How funds change benchmarks to flatter their performance

By LARRY SWEDROE

Securities and Exchange Commission (SEC) Rule 33-6988 requires mutual funds to disclose an “appropriate” broad-based stock market index to which they benchmark their performance. Specifically, funds are required to compare their past one-, five- and ten-year performance to that of a benchmark index they choose. The freedom of choice provides funds with significant leeway, allowing them to choose benchmarks that are easiest to beat.

For example, it is well known that due to problems of front-running changes to the index, the Russell 2000 has been a poor choice for an index fund to replicate, but a good choice as a benchmark for an active fund — over the period 1994-2021, the Russell 2000 returned just 9.5 percent, well below the 11.1 percent return of the S&P SmallCap 600 and the 11.2 percent return of both the CRSP 6-10 Small Cap Index and the MSCI U.S. Small Cap 1750 Index.

Another issue, one that investors are likely unaware of, is that surprisingly the SEC allows funds to change their benchmarks with little scrutiny or justification and “backdate” their performance relative to an index with a lower past return. Kevin Mullally and Andrea Rossi, authors of the July 2021 study Benchmark Backdating in Mutual Funds, analyzed whether fund sponsors took advantage of this flexibility to mislead investors. Their data sample covered actively managed funds over the 13-year period 2006-2018. Following is a summary of their findings:

• In 454 out of 766 benchmark additions (59 percent), the benchmark the fund chose was in a style that was not the most reflective of its investment strategy. For these mismatched cases, funds chose to add styles with benchmark returns that were 1.39, 6.51 and 8.83 percentage points lower than those of the best-matched style over the past one-, five- and 10-year periods, respectively.

• Funds added indices with, on average, 2.21 and 6.12 percentage points lower five- and 10-year returns than those they currently reported. The differences were statistically significant at the 1 percent confidence level.

• Funds deleted indices with, on average, five- and 10-year past returns that were 1.53 and 7.08 percentage points higher than the ones they retained. The differences were significant at the 1 percent confidence level.

• Funds added (dropped) indices with lower (higher) past returns, improving the appearance of their past relative performance.

• To improve the appearance of their relative performance to attract more capital, funds with lower past performance and lower past flows were more likely to change their benchmarks.

• Funds that changed their benchmarks were more likely to inflate their end-of-period returns and had less sophisticated clienteles. Specifically, funds that changed their benchmarks engaged in more “portfolio pumping” (purchasing large amounts of shares in existing positions shortly before the end of the reporting period to inflate prices and returns) and were more likely to be sold by brokers (research has shown that broker-sold funds have less sophisticated investors).

Mullally and Rossi found that funds strategically changed their benchmarks to improve the appearance of their relative performance. They concluded: “Our findings suggest that the information some funds present to investors about their relative performance does not appear to be reliable.” Sadly, demonstrating their naïvety, investors rewarded funds for backdating their past performance — funds that changed their benchmarks received positive abnormal flows in the next five years. This finding was consistent with that of Berk Sensoy, Professor of Finance at Vanderbilt University and author of the 2008 study Performance Evaluation and Self-Designated Benchmark Indexes in the Mutual Fund Industry. He found that about 31 percent of mutual funds have a benchmark discrepancy; that a fund’s flows are influenced by performance relative to the prospectus benchmark even when a benchmark discrepancy exists; and that even after controlling for performance relative to a mutual fund’s factor exposures, investors react strongly to performance relative to a fund’s prospectus benchmark index (i.e., the benchmark that a fund declares in its prospectus to meet regulatory requirements).

Thanks to Martijn Cremers, Jon Fulkerson and Timothy Riley, authors of the study Benchmark Discrepancies and Mutual Fund Performance Evaluation, published in the March 2022 issue of the Journal of Financial and Quantitative Analysis,” we have new evidence that supports the findings of Mullally and Rossi. Their procedure for identifying benchmark discrepancies was based on fund holdings. They first assessed which of several potential benchmarks had the most overlap with a fund’s current holdings. They then evaluated the overlap using the Cremers and Petajisto Active Share measure. The benchmark that generated the lowest Active Share (AS) was considered to have the most overlap and thus was the most appropriate benchmark. They also required that there be low overlap of holdings between the Active Share benchmark and prospectus benchmark — defining a fund as having a benchmark discrepancy if the “benchmark mismatch” was greater than 60 percent. Their data sample covered the period 1991-2015 and 1,745 unique funds. Following is a summary of their findings:

A substantial portion of prospectus benchmarks understate risk and consequently overstate relative performance — benchmark discrepancies are common (65 percent of observations had a prospectus benchmark different from the AS benchmark), and the prospectus benchmark typically understates risk when a discrepancy exists.
In 37 percent of the cases in which the benchmarks differed, the benchmark mismatch was above 60 percent, and 17 percent had a benchmark mismatch above 80 percent.
Among funds with benchmark discrepancies, the AS and prospectus benchmarks had substantially different returns. The average return on the prospectus benchmark was lower than that of the AS benchmark by about 1.46 percentage points per year (t-stat = 3.08) — funds with a benchmark discrepancy generally need less return to beat the prospectus benchmark compared to the return needed to beat the AS benchmark.
Funds with a benchmark discrepancy tend to be riskier than their prospectus benchmarks indicate and have higher expense ratios.
Funds with benchmark discrepancies perform better relative to their prospectus benchmarks because their prospectus benchmarks tend to understate net factor exposure.
Before adjusting for risk, funds with a benchmark discrepancy outperformed their prospectus benchmarks by 0.17 percentage point per year on average but underperformed their AS benchmarks by 1.26 percentage points per year.
A prospectus benchmark overstating the performance of a fund by 1.00 percentage point had the same impact on net fund flows as an actual increase in performance of 0.54 percentage point — performance relative to the prospectus benchmark is an important determinant of investors’ capital allocations.

Their findings led Cremers, Fulkerson and Riley to conclude: “Among funds with a benchmark discrepancy, the prospectus benchmark significantly overstates ex post performance.”

They added: “The difference in returns between the AS and prospectus benchmarks of funds with benchmark discrepancies is the result of differences in risk. When a benchmark discrepancy exists, the AS benchmark tends to be riskier than the prospectus benchmark, as indicated by the AS benchmark’s larger factor exposures. Larger traditional factor exposures (i.e., market, size, value, and momentum) explain about a third of the average difference in returns between the benchmarks. Larger non-traditional factor exposures (e.g., liquidity) in conjunction with the traditional exposures explain about 89% of the average difference in returns.”

Unfortunately, they also found that “investors respond to fund performance relative to the prospectus benchmark even when a benchmark discrepancy exists”.

Investor takeaways

Risk adjustment is central to performance evaluation. Therefore, investors should exercise caution when using prospectus benchmarks to evaluate fund performance because a significant number of funds understate their risks by failing to choose a benchmark that is the best fit.

Unfortunately, it also appears that many investors fail to adjust the performance of actively managed funds for risk, likely because they assume that the prospectus benchmark is the appropriate one. You now know that is a bad assumption.

One way for investors to better judge performance of actively managed funds more accurately is to utilise the regression tool available at Portfolio Visualizer.

For informational and educational purposes only and should not be construed as specific investment, accounting, legal, or tax advice. Certain information is based upon third party data which may become outdated or otherwise superseded without notice. Third party information is deemed to be reliable, however its accuracy and completeness cannot be guaranteed. Indices are not available for direct investment. Their performance does not reflect the expenses associated with the management of an actual portfolio. Information from sources deemed reliable, but its accuracy cannot be guaranteed. Performance is historical and does not guarantee future results. Index total return includes reinvestment of dividends and capital gains. By clicking on any of the links above, you acknowledge that they are solely for your convenience, and do not necessarily imply any affiliations, sponsorships, endorsements or representations whatsoever by us regarding third-party websites. We are not responsible for the content, availability or privacy policies of these sites, and shall not be responsible or liable for any information, opinions, advice, products or services available on or through them. The opinions expressed by featured authors are their own and may not accurately reflect those of the Buckingham Strategic Wealth® or Buckingham Strategic Partners®, collectively Buckingham Wealth Partners. Neither the Securities and Exchange Commission (SEC) nor any other federal or state agency have approved, determined the accuracy, or confirmed the accuracy of this article. LSR-22-236