Stumbling and Mumbling

Wisdom of crowds - a puzzle

Is belief in the wisdom of crowds self-defeating? This is one question raised by the "slump."
Put yourself in the position of an intelligent investor back in the spring. You know that the yield curve is a vastly better economic forecaster than economists are, and its inversion signals that a recession is quite likely. You know too that  there's a good correlation between industrial production and equity returns. So you're tempted to dump shares.
But then you remember the wisdom of crowds. Surely, you figure, the fact that the crowd - not just of economic forecasters but of other investors - isn't thinking of recession shows that one isn't likely. And the market must be discounting the possibility  anyway: price-earnings ratios were quite low back then.
So, you decide not to sell.
In this sense, belief in the wisdom of crowds proved self-defeating. In believing the crowd was right, you didn't sell shares. The upshot was that shares were left over-priced, because some information about the likelihood of recession - the opinions of those who believed both the yield curve and wisdom of crowds - could not be embodied in prices.
Because some people thought crowds were wise, the crowd - the stock market - proved not to be so.
Now, this doesn't show that the wisdom of crowds is logically incoherent. Its advocates accept that one condition for crowds to be wise is that individuals' beliefs be independent, uncorrelated. Which is exactly the condition violated in this case.
What it does mean, though, is that the idea's relevance might be greatly limited in practice.
The problem here is not the same as the well-known paradox of efficient markets - that if everyone believed markets were efficient, no-one would bother exploiting price-relevant information and so markets would be inefficient.
In that case, information lies around, but is unused.
In the case of the wisdom of crowds, information is actually destroyed - the bearish beliefs of those who expected recession were reduced by their belief in the wisdom of crowds. In this sense, belief is the wisdom of crowds is positively damaging.

(Not) forecasting tastes

The "slump" in share prices reminds me of a curious correlation - the strong link in recent years between equity returns and US economic growth. 
I say "curious" because this link shouldn't exist. Markets should see recessions coming and mark down share prices in advance of them; how hard can it be to just (pdf) look at the yield curve? So why should shares fall during recessions?
Yesterday's big price falls add to this puzzle. The papers say these were triggered by news of Citigroup's troubles.  But this won't do. Any fool could tell weeks ago that banks had yet to annouce the full losses on mortgage securities. And the dividend cut was anticipated long ago.
Nor is it plausible that the drop in December's US retail sales should come as a huge surprise.  The real surprise about retail sales is how strong they've been recently given the weakness of the housing market - up 5.2 per cent excluding autos albeit in nominal terms in the last 12 months. In this context, December's fall, as James says, is only mildly disappointing.
So, why should the market have fallen so much yesterday, and be so sensitive to contemporaneous moves in industrial output?
It can't be solely because American investors and especially equity analysts have an optimism bias that blinds them to the possibility of earnings shortfalls or recession. My chart shows that strong economic growth in the late 90s and in 2004-05 was as good for shares as weak growth was bad.
So, here's my theory. Investors are terrible at is predicting their future tastes; see this great paper (pdf).  They might - or at least should - be able to forecast economic  upswings and downswings. What they don't foresee is the impact these will have upon their appetite for risk. They don't see that bad economic times will cut their appetite for risk and that good times will raise it. The upshot is that shares fall in bad times even if such times should be foreseeable.
There's a lesson here beyond the triviality of short-term market moves. If people are bad at predicting their future tastes, even when billions of pounds are at stake, doesn't this suggest that our ability to make choices that make us happy might be severely limited?

The ideology of financial instability

Should we have a cabinet minister responsible for financial stability? Peter Franklin thinks so:

Devoting a Cabinet minister to financial stability is no guarantee of financial stability, but it would help and, if nothing else, would send a message to the anxious savers and pension fund holders of Middle England that we take their concerns seriously.

Matthew disagrees:

Just about the only way you could make things worse is to put a politician in charge. This isn't an area where there are many ideological issues at stake that should be decided democratically. It is, instead, an area that requires expert and experienced judgement.

I agree with Matthew that it's a bad idea to put a politician in charge. But I disagree that this is an area requiring expert judgment. The thing is, financial stability is an ideological question, on which experts disagree. There are three distinctive positions here:
1. Free markets create financial stability. Financial entrepreneurs invent clever ways to split risk into small bundles and sell it to those best able to bear it. State intervention can destabilize the system by introducing both inflation and moral hazard; if lenders think they'll be bailed out by interest rate cuts or loans from the taxpayer, they'll take excessive risk.
You might think this view is even less credible now than a few months ago. Not necessarily. Many of the losses on ABSs are losses of "internal assets", which are redistributions of wealth rather than aggregate losses of real wealth. And the stock market only looks unstable because you look at it every day. In the last two years, the All-share has risen at an annualized rate of around 5 per cent - in line with its average for the previous 105 years. The casual observer - and serious retail investors should be no more - would never guess there was a problem.
2. Markets are inherently, endogenously unstable, as Hyman Misky pointed out. Good times cause banks and investors to become over-exuberant and lend too much. When reality dawns, falling asset prices and tighter loan standards cause recession.
3. Markets would be a force for stability, if only they were better developed; this is Robert Shiller's position. If we had markets in occupational or industry risks, ordinary people could get better insurance against the economic risks they face. One job of government should be to encourage the "financial services" industry to stop being an oxymoron and start developing these markets.
To have a minister for financial stability in the absence of agreement on these questions is pure managerialism - the belief that the good judgment of someone in power can adjudicate between competing theories. And of all ideologies, this is surely the one with the least to commend it.

Black Monday: could it happen again?

Today is the 20th anniversary of Black Monday, the largest one-day fall in the Dow Jones index in history. Which raises the question: could it happen again?
If you believe returns are normally distributed, the answer - for practical purposes - is: no.
The 22.6% drop in the Dow then was (assuming annualized volatility of 20%) an 18 standard deviation event. And on a bell curve, the odds are against such events happening even once in the entire history of the universe.
But do returns follow a bell curve? What could we say to someone who says: "Returns are normally distributed - we were just very unlucky to see it in our lifetime."?
We could reply that lower standard deviation events are also more likely than a bell curve predicts; I estimated before the summer's falls that three standard deviation events are four times more likely than a bell curve predicts.
"Aha" our normalist might reply. "That's irrelevant. You can't infer the likelihood of 18 standard deviation events from three standard deviation ones. Indeed, probabilities must sum to one, so the fact that somethings happen more often than they should means that other things must happen less often. So perhaps 18SD events are even less likely than a bell curve predicts."
We could then try the wisdom of crowds argument. Options consistently price in a higher chance of a big fall than the bell curve says they should.
"This isn't wisdom" replies the normalist. "It's Charles Mackay's madness of crowds. It's a common cognitive bias to over-react to easily-remembered events. The memory of 1987 means deep out-of-the-money put options are over-priced. Look how many hedge funds have made money selling them down the years."
We could then point the normalist to this paper, which gives evidence that share returns are distributed as a power law with an exponent of 3. This predicts that a 10% daily fall should happen roughly once every 11 years and a 20% fall every 100 years.
"Come off it" replies the normalist. "Any fool can fit a curve to a sample. It doesn't follow that the curve will continue to fit the future data. That's just blind extrapolation."
I reckon, then, that all we're left with are Bayesian priors. It's not the data that tell us a crash could happen again, but common sense. We know that investors can herd, that there can be common mode failures in which disasters come not as single soldiers but as whole batallions, and that, well, shit happens.
This, though, means that "probability" is not something that exists independently, and is given by the data. Instead, it's something subjective that we impose onto the data.
Which in turn means that risk management is not - or at least should not be - narrow number-crunching. That's not science, but rather the appearance of science. Riccardo Rebonato shows why in the most readable book on financial risk you're likely to find.

When markets fail

It's been a bad week for prediction markets. First, the Nobel prize in economics goes to three guys who weren't in the betting. And then outsider Anne Enright wins  the Man Booker prize.
The wisdom of crowds is 2-0 down.
Does this mean prediction markets are over-hyped?
No. It just shows their limits. Markets work best when there's lots of dispersed information to aggregate - about, say, the state of the economy, election outcomes or the success of films. This paper provides some evidence.
But in these two cases, there was no such dispersed information. The decisions were taken by a tiny number of jury members  who gave no signals about their intentions.  Local knowledge - "Gordon Tullock's a great economist", "On Chesil Beach is selling well" -  was no help.
Crowds know lots, and should be used more. But they don't, and can't, know everything.

Yes, markets are irrational

I'm now convinced. Financial markets are inefficient. What's made me realize this isn't the credit crunch, subprime crisis or Northern Rock. It's more important than that - the betting on Charlotte Church's baby's name.
The thing is, the names of her baby - Ruby Megan - featured nowhere on Paddy Power's long market of names.  This represents a failure of prediction markets.
What's more, the failure is the result of a clear cognitive bias.
The favourite names among bettors show a strong bias. It's towards names associated with Charlotte and Gavin - Maria as in Ave, lots of Welsh names and rugby players. And it's underweight  in popular names generally. 
This is a failure of Bayesianism. It's the base rate fallacy. People over-rated specific knowledge (what might Charlotte and Gavin call their baby?) and under-rated background probabilities; what do young people generally call babies?
Had the betting been based merely on background probabilities, Ruby would have been fourth favourite. That it was nowhere near proves the irrationality of markets.
Eugene Fama was wrong.

Stock-picking doesn't work

This chart is revealing. It shows the performance of one class of hedge funds, according to Hedge Fund Research.
Which class? Not macro. Not distressed securities. It's equity market neutral. Yes, neutral. Such funds, in theory, should take short positions to, well, neutralize stock market exposure.
But when they most needed neutral exposure, they didn't have it. Between July 3 and last Friday, neutral funds lost nearly as much as MSCI's world equity index - 5.7% against 7.8%.
One reason for this, as Eric says, is that changing correlations and volatilities screw up the calculations behind stat arb trades.
Another reason, I suspect, is that the turnaround in markets has hit momentum investing hard, as the stocks that benefited from high liquidity (such as commodity stocks and takeover targets) have lost from its withdrawal.
Another feature of the chart is just how low the returns are to neutral funds even over the longer-term. In the four years to early July (the peak), this index returned barely 12% - less than UK cash.
As hedge fund indices are probably not  downwardly biased measures of return, this is pretty feeble. Even if the stock market is irrational, it doesn't follow that you can easily out-perform it.

Nothing unusual

Guido asks a very silly question:

FTSE 100 is down 10% this month and big UK mortgage lender Northern Rock has halved in price since the beginning of the year. What are the political ramifications for Gordon? Well didn't he tell us he had abolished boom to bust?

The political ramifications are approximately zero.  For one thing, Gordon never promised to abolish boom and bust in financial markets; not even he is that stupid or arrogant. And for another, we learned in the tech bust (and indeed in the aftermath of the 1987 crash) that big falls in share prices can have remarkably little impact upon the wider economy.
More importantly, a 10 per cent fall in the market is just no big deal.
Let's assume market volatility is 20% a year, its long-term average; yes, the market was less volatile than this between 2004 and June 2007, but few expected that stability to last.
20% annual volatility implies monthly volatility of 5.8%, assuming zero serial correlation. So, a 10% fall in a month is a 1.7 standard deviation event. Even if we ignore fat tails and assume a normal distribution, this is the sort of thing we'd expect to see 4.5% of the time - one month every two years. Allowing for fat tails implies a higher probability.
This is a pretty ordinary event. It might excite the chuntering class, but not serious people.

Chebyshev and chickens

Felix Salmon asks what the hell David Viniar, Goldman’s chief financial officer, meant when he said:

We were seeing things that were 25-standard deviation moves, several days in a row [last week].

Clearly, if Mr Viniar is thinking in terms of bell-curves, this is just cretinous. Even a 9 standard deviation event is unlikely to have happened even once a second since the universe began.
Mr Viniar can't be that stupid. He knows financial returns are not normally distributed, but have fat tails; extreme events are more likely than a bell curve suggests.
More likely, he's thinking of Chebyshev's inequality. It says that in any data sample,  no more than 1/k² of the values are more than k standard deviations away from the mean. On this view, a 25 standard deviation event is a one in 625 probability.
Does this make Mr Viniar look less stupid?
Not really. If we define "several"  as  "three", then the chance of getting three successive 1 in 625 probabilities is roughly one in 244 million.
No, however we cut it, Mr Viniar is talking wibble. He just under-estimated risk.
There's one very stupid way of doing this. Imagine you're a chicken. Every day, the farmer feeds you. After a while, you figure: "My returns from the farmer are pretty stable, as I seem to get roughly the same amount of corn every day. Being a chicken  is a low-risk business."
The following day, the farmer breaks your neck.
Any hedge fund who took default risk (say by holding CDOs) and boasted about its Sharpe ratio based on post-2003 returns would have done much the same as this; you can't measure default risk by looking at past returns.
I had assumed that no-one was stupid enough to fall for this trick; one reason why I was underwhelmed by Taleb's The Black Swan - which laboured similar points - was that I thought he was just stating the bleedin' obvious.
But perhaps I was wrong.

Why the subprime problem matters

The difficulties in US sub-prime mortgage markets might have a profound implication.
No, I don't mean it'll cause a sustained economic slowdown; it's remarkable just how well American consumers' spending and confidence has held up recently.
Instead, the turmoil undermines the simple-minded justification for free markets.
This view says free markets can spread risk effectively by splitting it into small parcels and selling it to those who are best placed to carry it. On this view, hedge fund bosses' big pay is justified because they skilfully assess risk and, in doing so, reduce its price and so cut borrowing costs, allowing for higher economic activity.
If this view were right, defaults in sub-prime markets should have had little effect. The market is tiny compared to all capital markets, and the risk is easily spreadable.
So, why have stock markets dropped so much? One possibility is just natural over-reaction. Another is that this self-congratulatory view of markets is wrong. Markets took sub-prime risk not because they parcelled it out wisely, but simply because punters were over-exuberant. Many hedge funds made money simply by taking blind bets that happen to have paid off; they're just like navvies who sometimes get lucky in Ladbrokes.
Unlike the navvies, though, there's someone around to bail them out when things get bad.
It looks like Hyman Minsky knew more than the neoclassicals.