Evidence, overconfidence & the crisis

One of the strongest complaints about the coalition is that many of its policies - for example, on taxes, EMA, or troubled families - are not based upon evidence.I wonder whether this failure is connected to the financial crisis.

To see what I mean, start from Andrew Lo's point that financial markets are neither efficient not inefficient but rather adaptive. Profitable opportunities regularly emerge, only to disappear as traders recognize them and bid them away.

To take just one example, in the 1980s it became increasingly well-known that smaller stocks had for years beaten the market. This sparked increased buying of them, but such buying pushed their prices up to a level from which subsequent returns were poor.

This means that evidence-based rationality can be counter-productive in financial markets. If you wait until there is huge evidence that a particular strategy works, it's likely that other people will also be aware that it works, and their buying will have moved prices against you. This was the fate of those who bought small caps in the late 80s.

In this sense, the trading mentality is the antithesis of the academic mentality*. The academic is cautious, proportions his beliefs to the evidence and wants more research.But the trader who waits for strong evidence will have missed the profits. The academic wants to be right, the trader wants to be first.

What's true of trading is, to at least some degree, true of entrepreneurship and business generally. The entrepreneur acts on hunches, often against the prevailing wisdom. There's a reason why many business-minded people feel frustrated in big bureaucratic firms that demand more market research, and why businessmen and academics often have a mutual incomprehension.

Which brings me to my point. If the trader's mentality enters politics - either because politicians have a business background or because they are influenced by association with business rather than academia - then we will have policy-making without an evidence base. What's appropriate in one sphere is not appropriate in another.

But what's this got to do with the crisis?

Plenty. To trade in adaptive markets, you must act upon a shaky evidence base, and this requires overconfidence. I suspect that if there is one character trait which links traders, it is not greed or amorality but overconfidence.

And, of course, it was overconfidence in various forms - in RBS's belief it could pay billions for ABN Amro, in Northern Rock's reliance on wholesale finance, in other banks' holdings of mortgage derivatives - which triggered the crisis.

If I'm right, or nearly so, there's an implication here. It's that the crisis was due not simply to individuals' failing, but rather was the result of the over-application of the sort of mentality that it is actually often necessary to have in financial markets.In other words, the crisis arose from the very nature of markets selecting for particular character traits**. The qualities that help a species thrive can also - when the environment changes - also hasten its demise.

* I'm speaking here of stereotypes. Some academics (those who want celebrity!) have a trader's mentality, and some traders are more academic than others.

** Yes - this is a variant of Minsky's financial instability hypothesis.

Comments

[Will Davies]

"the trading mentality is the antithesis of the academic mentality"

Yes. This is exactly why Hayek favoured markets over social scientists (including most economists). Markets would act as massive computational devices, as an *alternative* to theory. That does, however, require people to plug themselves into markets in a strangely instinctive, non-reflexive fashion. As you say, hunches and urges have to feed into the price system, without having been first converted into considered judgements.

Contrary to what you suggest, one Hayekian analysis of the crisis might be that there was too much rationalist analysis going on, too many theories at work, too many economists adding models and complexity to a system that depended on very simple forms of human behaviour. Donald MacKenzie's analysis of how Black Scholes became 'performative' and then 'counter-performative', as it became more closely entangled in derivatives markets, would be a case in point.

Ultimately, the Libor scandal shows that the banks ended up engaging in precisely the type of self-conscious, expert, reflexive activity that Hayek was most afraid of, namely planning.

[Stian]

Overconfidence is much more useful in politics than in financial markets. To see why, consider the difference between traders and politicians.

Any single trader's overconfidence won't move the market. Sure, traders can afford to be overconfident because they don't have to bear the full costs of failure. But their confidence in their own success has limited effect on whether their bet will pay off.

But for politicians, optimism makes them more liekly to succeed. Politicians' job is to lead. They need to convince people that they are doing the right thing, and motivate collective action. The most successful politicians exude something like Steve Jobs' reality distortion field - they seem to succeed in part through force of will. Of course, this has been researched (Seligman's work on how optimist presidential candidates beat pessimistic ones springs to mind).

The challenge is how to get politicians who are optimistic enough to be effective leaders, but humble enough not to commit us to daft irreversible policies. (One way of course is democracy.)

[kraut]

There doesn't seem much of a risk of a business mentality taking over politics. All parties are now dominated by career politicians who have never had any meaningful experiencs outside of politics.

[Metatone]

Will Davies makes a poor attempt to deflect blame from Hayek for this latest round of crisis.

The reality is that markets set up according to Hayekian principles almost always collapse into gamed pricing.

[Account Deleted]

I think one aspect of the adaptive market hypothesis that is relevant to the LIBOR affair is the idea that survival trumps profit. This certainly seems to have been a factor in Barclay's manipulation in 2008.

The behaviour of "rogue traders", such as Leeson and Kerviel, taking ever bigger positions to cover losses, would also seem to support this.

The visceral fear of the U-turn is perhaps an example of this psychology in the political domain.

[Andrew]

Chris, you are obsessed by this one cognitive bias of overconfidence.

Performing any activity (eg life) with insufficient evidence certainly does not require overconfidence. Why should it?

You are confusing the sorts of character traits that may be selected in certain games with character traits necessary to operate. Do you think the most successful traders are the overconfident ones or the ones that judge probabilities accurately?

[Will Davies]

Metatone - I have no interest in rescuing Hayek or deflecting blame from him. Frankly, the Hayekian ideal is impractical and anthropologically naive. I was simply highlighting that there is a Hayekian analysis of the crisis, and quite sound reasons to suggest that the 'neoliberalism' that have had over the last 20 years or so is not quite the one that Hayek would have advocated. It's difficult to imagine a market system (especially financial markets) built purely on hunches/desire/instinct, without more dispassionate forms of theory and expertise - but I think that an excess of economics/models is one plausible explanation for the crisis, given that it is ultimately one of excessive complexity.

[Neil]

"If the trader's mentality enters politics - either because politicians have a business background or because they are influenced by association with business rather than academia - then we will have policy-making without an evidence base. What's appropriate in one sphere is not appropriate in another."

This is a very promising line of thinking that deserves to be explored further; it is entirely possible to characterise the fundamental left-right divide as a disagreement between academics and businessmen.

The question then becomes: Is politics more like business or academia? If what's appropriate in one sphere is not appropriate in another, it's useful to know what sphere you're actually in. Presumably David Cameron would argue that in public policy, it is more important that your policies be successful than that they be provably correct...