Is your fund's performance statistically significant?

Some funds seem to outperform the market, but appearances can be deceptive. Before trusting those results, investors should ask whether a fund’s performance is statistically significant.

In simple terms, is the outperformance due to genuine skill, or is it just random chance? According to indexing advocate MARK HEBNER, most people do not know how to make that distinction and that can lead to costly decisions.

KEY TAKEAWAYS

1. Market-beating funds often rely on luck

A single year of strong returns can mask years of underperformance. If a fund delivers a big return one year but lags behind the benchmark in others, the apparent success may be the result of random chance, not genuine skill.

2. Statistics are essential for assessing funds

Understanding variability and statistical significance is crucial when evaluating performance. Investors without a background in statistics may mistake luck for skill and end up paying high fees for no long-term benefit.

3. Peer-reviewed research is your safest guide

Unlike marketing material or commercial fund data, academic research is rigorously vetted before publication. That makes it a more reliable source for understanding how markets work and where real value lies.

RELATED CONTENT 

https://www.evidenceinvestor.com/post/the-persistence-of-fund-performance-what-the-evidence-shows 

https://www.evidenceinvestor.com/post/does-your-fund-manager-punch-walls 

https://www.evidenceinvestor.com/post/how-do-active-funds-perform 

TEBI ON YOUTUBE

Have you visited the TEBI YouTube channel lately? There’s already a wide selection of high-quality videos on there. Why not subscribe and be one of the first to see our latest content? You'll also find our videos on Instagram and TikTok.

TRANSCRIPT

Robin Powell: We often read about funds that have outperformed the market, or, to use a technical term, delivered alpha.

But investors always need to ask: Is that fund’s performance statistically significant?

In other words, how sure can we be that it resulted from genuine skill and not just random chance?

Mark Hebner: What people don't pay attention to. You don't see it in Morning Star or Leper or any of the services that track mutual funds is the variability of that alpha. 

So one year they made 5% and then they went 3 or 4 years -1% to the benchmark. Right. But that one year with the five or even higher, maybe 10% was their lucky outcome.

And even though they had multiple years after that earning what they should earn, which is below market returns, it appears that they had six years of great performance. 

RP: In investing, distinguishing between what statisticians call a chance outcome and a systematic outcome is very complicated.

Only a small proportion of financial advisers, let alone investors, have the statistical knowhow required to even attempt it.

MH: It's what I call the language of risk. Risky. You have to speak this language to fully understand it. And just like legally is, you know, if you don't know all the legal language, you probably shouldn't be representing yourself in court. 

Well, if you don't speak risky, you shouldn't be representing yourself in the public markets, okay? You have to have a better understanding or hire somebody who understands these statistical principles. 

And I think that's probably the number one problem for people as investors is they just want to rely on their gut. And then instead of at least a reasonable knowledge of statistics. 

RP: But of course, this is not to say that all statistics are misleading.

As Mark Hebner explains, the data contained in peer-reviewed academic research are generally the most reliable.

MH: An academic paper has a series of what are called referees that check these statistics before an academic is able to publish the paper. It is. It goes through a rigorous test and checking by other academics before it can be published. That's peer reviewed research. Huge. Huge idea. 

These things have been tested and checked by other almost competitor academics. These academics are competing really to publish, you know, really high quality research. 

And so nobody, no academic wants the other academic to get away with data mining. Small sample sizes, non statistically significant data. 

RP: In short, if you don’t understand statistical significance, you really shouldn’t be choosing actively managed funds.

Ideally, find yourself an adviser who does understand it and who does have a good grasp of academic finance.