CFA SF Panel: Manager Selection Best Practices 10/25/2012 Andy Isieri Callan Associates (could not attend.

d. presentation given by host) Lou Fernandez Hall Capital Partners (replaced Eric Alt) Chuck Stucke Guggenheim Kelly Dupont Probitas Partners Andy Isieri - Callan Associates (Andy could not attend, but his presentation was walked through by the event host) Callan approach: Manager selection does not come first. Recall CFA L3 material, must first create IPS, establish target rate of return,, manager selection is 7th step Repeatable success, even predictable if you will 4Ps of what drives investment performance people philosophy process price, to a lesser extent Must guard against performance chasing Pay attention to mean reversion Risk: pay attention to the ability vs. willingness of your client to take on risk Avoid falling in love with managers, and also avoid being a hater. Example: Shunning all quant strategies after 08 On manager research: People biggest variable Culture motivation, retention, recruitment Bad strategies are easier to identify Evaluate drivers of performance, not performance itself Rebalance your losers Be curious investigate (ex: Colombo detective) More Callan approach: Quant analysis first, then qualitative -> holistic approach First thing Callan does is create 250 page analysis of statistical ratios (Sharpe, sortino, standard deviation, etc) [Differs greatly from other presenters who prefer qualitative analysis first, before digging into numbers] Qualitative: ask why. Talk to all levels of the operation. Past employees are particularly useful, but only if they have been gone for a year or more. Then they start gushing w/ info

Lou Fernandez Hall Capital Partners We differentiate ourselves on Integrated approach Example: a public equity fund may have a PE-fund like structure. overlap 22 members on research team, about 5 people per asset class Absolute Return/Complex Credit Real Assets Capital Markets Private Equity Seeing opportunity now in complex credit strategies Total return approach as opposed to income only Philosophy: Partner with best managers, emphasis on partnership/relationship Fundamental research bent Risk via diversification Think about risk as protection on downside Not so worried about volatility, its not a good measure of risk More worried about achieving point to point return. Must have ability to withstand interim volatility Asymmetric risk/reward profile Prudent concentration based on managers underlying deep research and knowledge of companies as opposed to owning the market Long term perspective. (patient capital, ability to withstand volatility) Research process: 10k funds in database 2.6k materials sent 500 initial meetings 110 in due diligence 30 approved (incl. 19 add-ons) (18 terminated) (incl. refusal of investing in new vintages of existing funds) Deep due diligence equals conviction Seeking durable strategies Also seeking broader opportunity set, not too niche. Must be able to navigate many market landscapes Evaluate Source maintain relationships with closed fund managers for entry when they are open Equity

Analyze trend over past few years is to focus on operations not just investment side. Also look at performance during crisis Monitor open dialogue Exit/recommit Different approach from Callan: Hall prefers qualitative first, and then quant Qualitative topics: -Incentives for manager compensation -Succession planning: industry trend of increasing short term performance pressure is causing many managers to leave the biz. Transition planning is a big issue Risk Management: Asset allocation is the first line of defense Liquidity Capital preservation Credit quality Counterparty exposure Summary Broad opportunity set, not too niche Open dialogue is crucial. Access to information, allows them to keep conviction Durable, repeatable strategy Long term perspective Chuck Stucke Guggenheim Partners 6 lines of business at Guggenheim Research/DD Process: [Shows slide of funnel filtering entire fund universe to selected funds most popular slide of manager selection ever] has list of 50 funds they believe are top notch. Also offered as a FoF product quantitative and qualitative decision making process Really need to understand strategies. Many investors invest w/o doing their homework Example: option writing strategy embedded across other asset classes, say leveraged fixed income. Might not be exposure you expect Sharpe ratio is extremely dangerous, so are many other statistical ratios 1st flaw is that it uses standard deviation to measure volatility (ie to describe risk), but hedge fund returns are NOT normally distributed. Should not be used in same sentence Managers: do not be afraid to argue why your sharpe ratio may not be as high Risk parity. Cannot talk about fat tails & normal distribution characteristics in same sentence with hedge fund returns (because not normally distributed) We like some CTA/macro funds

Within quant funds, we categorize as Mean Reverters or Trend Followers Mean reverters are short volatility There are more funds available than there are stocks! Focus on the primary factors of the funds Operational risk, becoming more important. (analyze Investments/ops/legal/transactions) Registered products are NOT safer vs. private market products Example: Due diligence on Sub advisor to a multibillion dollar fund Site visit to sub advisor: shop was above a 7-11 at strip mall [says cares about Guggenheim image, especially in NY, cannot be invested in such fund] Example: a $2B fund was using an administrator nobody in firm had heard of. Started asking around, other people had heard of them and never heard of trouble, but kept digging. Turns out that this fund was 50%+ of revenues for the administrator. Not ok. Very biased relationship Understand financing dynamic of funds. Who provides them capital, for margin/leverage? During crisis some funds had locked in loans at rates lower than what the banks were paying, so the banks took losses to write those loans. Had to monitor solvency of banks Daniel Kanheman & behavioral biases [possibly the most interesting aspect of discussion] Psychologist who won Nobel Prize in economics In 1970s wrote paper on Prospect theory, financed by a Guggenheim client, one of the Guggenheim foundations 1 Analysts operate an inch wide, and a mile deep. After all of their research they tend to believe their work. Because of this, Guggenheim separates all research and investment decision making functions 2 Endowment effect. Say you buy a bottle of wine for $60 and it goes to $60,000. Will you sell it, buy another one, or drink it? Kanheman found that most people do nothing. For example, Guggenheim several years ago had tremendous conviction about an overheated PE market. This belief led to Institutional Investor writing a piece about this view. They even had a chance to sell into the secondary market for an 8-10% discount. But, they did not take this chance, and ended up taking far greater losses. They had conviction in a belief, but froze when presented with a chance to act on it. This is endowment effect in action Hopefully understand this mistake well enough not to repeat it

Kelly Dupont - Probitas Probitas is a placement agent, helping gps raise funds PE is an inefficient market, does not lend itself well to statistical analysis at all [shows chart of VC funding by year since about 1997] 1st bubble 2000. VC funds > Lbo funds (ie vc funds drove the funding growth) bubble burst: in 2Q 2003 there was negative fundraising in VC more money was given back than was raised nd 2 bubble 07, 08. LBO funds > VC. (LBO funds drove funding growth) Bubble burst: now we are in new normal, will stay below 06-08 levels for long time PE - Key Differences -Liquidity: secondary market at 10-15% discount usually. During crisis some funds, even top names were selling at 30cents on dollar in 2Q 2009 Dirty secret of PE: Funds really have about a 14 year life -Active management: board seats: determining strategy, hiring/firing management -Concentrated: 10-15 companies -Valuation: very subjective. Also delayed 1qtr lag, plus received 45 days late. Example: Blackstone and KKR held same asset, but they valued them with a 25% difference -Definition of volatility -Inefficient asset class: wider dispersion between funds, 2000bps between top and bottom quartiles. (two thousand bps) -> In PE, manager selection drives returns Key concerns -Benchmarking returns: PE benchmarks are horrible. Do not trust any of them. The Cambridge ones have selectivity issues, because data is only from their funds. Prequin is okay but not good either. -Attribution: Whose track record is it? Ex. A firm was trying and trying to raise Fund VIII, but struggled and eventually gave up. Why? Senior turnover during fund V or VI, so the previous track records did not apply to new fund. -Unrealized returns: how believable are they? Ex. Blackstone/KKR: same asset valued 25% differently. Hope does not always spring eternal -Track record volatility: Skill or luck? Is it sustainable? Ex Willis Stein was a firm a while back in Chicago. Fund I was $300m, fund 2 was $600m, fund 3 was $1.8B. Fund 3 made the firm collapse. Could not handle size. -Style drift -team stability: Will the people who generated returns still be around? -Ex In Asia, emperor syndrome. 1 guy in charge runs show, if he leaves its over. Or other critical members may leave if they want to run their own shop/get more $ -Ex Golf handicap test. If you can find the golf handicap of the manger online and its low, then he probably wont be around too much

-Ex Junior analyst test. Bring jr guys to meeting, have them go to lunch with jr guys from the managers firm. See what is said when chief is not around. Meet and discuss afterwards. -Ex SmithJones [generic name used] PE shop invited a potential investor to the annual meeting. Investor noticed partners Smith and Jones never spoke to each other, at all. Felt funny about this and decided not to invest. Few months later Smith and Jones went their separate ways and firm closed down Terms and conditions: see ILPA guidelines. Decently helpful. Incredibly important because once you are locked in it can be very difficult to change terms Q&A Q for Kelly: Thoughts on Kauffman foundation report on VC? A small I banks no longer exist for growth financing.[missed last part of answer, sounded like leads to increased risk for funds?) Q: Do you use a scoring system/rating process/watch list for managers? Lou: We have a pipeline we watch very closely. No formal scoring. Team based decisions/discussions Kelly: Triage system for younger team members to learn what to focus on For 10-12 deals, 1-2 may be emerging managers Chuck: We actually do use a scoring system for each category (ie investments/ops/legal) Came about from working with Danny kahneman Disaggregating decisions into smaller pieces leads to better decisions Spent many hours debating this topic. We use 1,2,3 scoring system (not 1-5,or 1-10) Q: How do you size up HF allocations? Chuck: HF is not an asset class. We allocate based on risk profile. Example of what to look at: equity l/s fund running gross 125, Beta 1 Also, balance sheet based. Evaluate effect of managers balance sheet on ours. Very cognizant of investing in their business Lou: Depends on fund size, leverage, potential for risk A more flexible generalist absolute return might be 5-6% Niche strategy could be only 1-2% Q how important is skin in the game? How much is enough? Kelly: in the old days, traditional PE GPs put up 1%, LPs 99%-->has now changed We want to see substantial portion of net worth. Enough to hurt if it goes bad Lou: Having vast majority of asset invested can also be bad. Example: will be more conservative if supposed to be risky/levered Chuck: In US fixed income, we prefer they own their business and care about track record. In HF, like to see a lot invested In PE, we are frustrated with the lack of skin in the game This is a real estate developers mentality which we dont like Kelly disagree with chuck. Thinks pe is putting enough in skin in the game

Q: Discuss evaluating emerging managers without a track record Chuck: We do a lot of day 1 allocations. For equity, WHO DID THEY APPRENTICE UNDER? Not what school/what degrees Apprenticeship is the single greatest predictor For quant, we evaluate the fund infrastructure Lou: not too many emerging manager allocations, but we look at previous positions (were they a previous PM or partner as opposed to just an analyst?), look at how they grew up/must get to know them well The landscape for setting up new funds is getting harder and harder with regulatory burdens. They must be excited about both running their own books AND running their own business. Kelly: Need investment experience. Also look to see if previous partner/PM position, not just analyst. This experience is important for decision makingsay they need to fire management for activist strategy, there is no substitute for experience Chuck: also, its said most managers are most productive in their first six years. The research overstates this due to selection bias. We bought graveyard data for a previous HF index. Shows selection bias adds 200-300 bps /yr to index returns. It does not show the funds that crashed and burned. Kelly: Recall the difference between top and bottom quartile PE funds. Its not about allocation, its about fund selection Q: termination process/mentality/judgment Chuck: obvious terminations: breach of integrity/loss of profitability. -It's hard to establish if over/underperformance is due to skill or luck, especially since the quant work is <20% of what we do -We have studied this topic: Allocating to underperformers enhances returns over time. Hypothetical: if you could add to underperformers monthly/quarterly, you would add 1% to your performance annualized Lou: turnover at the senior level often causes trouble -Shows importance of close dialogue to observe changes -Example: PM hires team to do work he used to do. Begins to delegate work instead, things will change. Managers are not often surprised when terminated because they see changes too Q: PE returns are driven by manager selection according to Kelly. What about on HF sideAA or manager selection?

Lou:

-Equity L/S more mgr selection -Absolute strategies see more AA type decisions because many managers identify similar situations Ie many similar credit funds will perform the same

Chuck: -Equity selection is skill based. This depends on PM so Mgr Selection -Credit/relative value is more market structure driven Maturation of technology tools has wiped out mgr selection here Example for 30k you can buy same tools same screens etc -New regulations favor large credit managers now. Before the large players would struggle against mid sized, but rules have changed Q: growth of liquid alternatives? Kelly: publicly traded fund vehicles (fund or FoF) are available. they end up trading in line with the market. (These products are different from the mgmt company going public, say KKR or Blackstone) -Every year we have a survey. 80% of clients not interest in liquid alternatives. 10% say we used to invest in them and do not any more Lou: there is an Asset Liability mismatch with them. HF replication based on factor models Chuck: some areas could work well, other they will fail -Example: activist funds needs to accumulate shares to unseat board. Cannot disclose positions so will never be able to be liquid. -Also depends on SEC leverage/diversification restraints if public/liquid. -Not all leveraged funds are inherently bad, many are good, but could be hurt by additional regulatory burden