Burton Malkiel: Why it took so long for passive to triumph

Posted by TEBI on December 28, 2023

Burton Malkiel: Why it took so long for passive to triumph



BURTON MALKIEL’s A Random Walk Down Wall Street, first published in 1973, was one of the first books to identify buy-and-hold indexing as the most logical option for most investors. 

The theoretical foundations for passive investing had been laid in the 1950s and 1960s. But it wasn’t until the turn of the century that index investing really started to take hold. So why did it take so long?

That’s one of the questions addressed in the second part of our exclusive three-part interview with Burton Malkiel. The interviewer is New York City-based analyst and financial writer JON JACOBS, to whom we are very grateful for allowing us to publish extracts from it.

If you missed the first part of this interview, you can catch up here


Please note that this transcript has been slightly modified for brevity and clarity.


Jon Jacobs: Professor Malkiel, please explain how the development of portfolio theory from the 1950s onwards drove demand for a wide range of index services. 

Burton Malkiel: Basically the answer is that the market portfolio is on the efficient frontier and other portfolios are inefficient in the sense of portfolio theory, in that they lie under the efficient frontier. 

In fact, even the S&P 500 is not the right portfolio. What you want is the entire market portfolio. And this has led to some change in thinking. What’s been in my books is that I don’t want you to buy an S&P 500 fund. I want you to buy a total stock market fund. And so something like MSCI Total Stock Market or the indices done at the University of Chicago, the so-called Crisp (CRSP). You see, CRSP indices are really the ones that lie on the efficient frontier.

The S&P 500 doesn’t. And presumably the Dow Jones Industrial Average would be worse. The Dow has all kinds of problems, but as a 30-largest-stocks-in-the-country portfolio, it lies further inside the efficient frontier than the S&P. So the problem with the Dow is, first of all, because it’s insufficiently diversified, it’s dominated by a portfolio on the efficient frontier. And it’s also a price weighted as opposed to a value weighted portfolio. And this would mean that you’d have to continuously rebalance it, incurring transaction costs. And incurring taxes, because you’d be realising in the rebalancing, probably short term capital gains. So it would be a very tax-inefficient portfolio. 

Widespread use of fund performance statistics derived from the CAPM — Sharpe Ratios, alphas and so on — began in the 1970s. Why did it take another 20 years or more for passive investing to catch on, even though it’s a logical consequence of those same theories?

First, a note on CAPM: it really did take over for ten or 20 years, but it’s not that popular today. Today, factor investing is popular. Today we think risk cannot be explained by beta. There are many different factors: profitability, quality, size and so forth. 

But clearly for a period it was thought it was very easy to measure risk, and beta was the way to measure risk. I think that as you measured risk and you were looking for these alphas, it was found that the alphas were ephemeral. If you had a fund that had an alpha in one year, it might just as well have the negative alpha in the next year. There was no persistence to the alphas. 

And that in turn led to people saying, well, maybe we should just buy the market and not even try. And then later I would say the other big push was from Standard & Poor’s SPIVA studies. 

You’re saying that in the 1970s, only due to the influence of the CAPM, did performance measurement become mathematically rigorous?

Exactly right. 

So then the proof that they still weren’t outperforming became much more convincing and accepted, and driving people toward indexing. Is that right?

I think that’s right. But I would say the accepted wisdom today is that relationship between return and beta is much too flat. It doesn’t work the way the CAPM model says. But I think the sequence was, you did start to try to measure performance in a rigorous way. And as you did that, more and more people saw that the emperor just doesn’t have any clothes.

That’s a very good way to formulate it. Apparently the SEC didn’t even mandate that mutual funds publish returns — again, let alone risks —  even returns, until sometime in the 1970s. 

And also what many of the active managers would do is publish returns pre-expense ratio. And also nobody thought of anything about taxes. In a year like 2022 when the market’s down 20%, how many people are just so surprised that they got a 1099 on their mutual fund showing that you owed a capital gains tax? Because the fund sold something at a profit. And even though the fund was down, you got a share of the profit. 

I’m wondering if the advent of portfolio theory itself might have helped nudge institutional asset allocations from bonds toward equities. These theories created an intellectual framework that justified and facilitated allocating professionally managed assets largely to stocks. 

I would say two things. Number one, portfolio theory showed you a way of reducing the risk of holding equities by holding portfolios rather than individual stocks.

The other factor during that period was that we had fixed interest rates held to very low levels during World War II to finance the war. And there had been a continuation of that into the early fifties. And then you had a period from the early 1950s into 1980 when interest rates went from very low levels into the Volcker period when long-term treasuries had double digit yields. During that whole period, interest rates rose and bonds proved to be terrible investments because as interest rates rose, their prices went down. 

I would say it was a combination of those two things. That portfolios of stocks were not nearly as risky as buying one or two individual stocks. And number two, that bonds were in a secular bear market.



Jon Jacobs is an award-winning financial writer who develops thought leadership content for asset managers and other financial institutions. He is currently gathering material for a book about competition among stock index providers.



In the third and final part of this interview, Jon Jacobs asks Burton Malkiel about how managerial decisions made by providers of both stock market indices and early index-based investment products shaped the growing industry of index data and related analytics in the 1970s and beyond.



Is it time to ditch the value factor?

Three disadvantages female investors face

Why fund industry marketing is so seductive



Through our partners at Regis Media, TEBI provides a wide range of high-quality content for financial advice and planning firms. The material is designed to help educate clients and to engage with prospects.  

As well as exclusive content, we also offer a wide range of pre-produced videos which explain how investing works and the valuable role that a good financial adviser can play.

If you would like to find out more, why not visit the Regis Media website and YouTube channel? If you have any specific enquiries, email Robin Powell, who will be happy to help you.


© The Evidence-Based Investor MMXXIII


How can tebi help you?