What sort of evidence, then, can investors rely on? In short, it’s academic evidence. But even then you need to be on your guard. For us there are four important questions you should ask when presented with anything purporting to be research on how to invest.
Is it genuinely independent?
New studies are being published all the time which appear to support a particular investment strategy or course of action. But all too often these reports are produced, or else commissioned and paid for, by companies with a commercial interest in publicising the outcome. Most academics on the other hand are truly independent in that they don’t have an agenda or a point to prove. Instead they leave it to financial practitioners to act on their findings or not.
Is it based on robust data analysis?
We all know the old adage about lies, damned lies and statistics. It’s true, abuse of data can be very misleading. Often findings are based on too short a time period or a sample that’s too small. Sometimes the fund industry ignores survivorship bias; in other words, it overlooks those funds which performed so poorly that they no longer exist. Other times it compares returns to the wrong benchmark, or it quotes performance figures before the full impact of fees and charges. Sometimes it simply gets its maths wrong.
Has it been peer-reviewed?
To test whether their findings are reliable, academics publish their research in credible academic journals. This gives other academics the chance to agree or disagree on whether the results are sound. Again, caution is required — there are journals that are less credible than others — but evidence that has been properly peer-reviewed should carry far more weight with investors than evidence that hasn’t.
Have the results been reproduced?
The fourth and final characteristic of findings you can depend on is that they’ve been tested across multiple environments and timeframes. There is some disagreement on the extent to which academic finance is properly scientific. Asset prices, for example, never move in exactly the same way as they have done in the past. However there does need to be a strong element of repeatability to demonstrate that the findings of a particular study weren’t just down to random luck or else reached through “data mining”.
Most of the evidence that we at The Evidence-Based Investor rely on passes all four of those tests. In other words, it’s independent and it’s based on robust data analysis; furthermore, the findings have been peer-reviewed and have also been reproduced. We would urge all evidence-based investors to insist on those same exacting standards.