On wishful thinking

David Aaronovitch in the Times has a nice piece on how people fall for conspiracy theorists and conmen:

What a fraudster, a fantasist or a hoaxer needs to be able to survive, is to fall among people who are anxious to believe.

This urge to believe is not confined to extreme cases. It's much more common than that.

A while ago, Guy Mayraz did a neat experiment at Oxford. He asked subjects to use charts of the price of wheat to predict future price moves. Before doing so, he randomly divided them into two groups: "farmers" who would profit from a rising price, and "bakers" who would profit from a falling one. He found that farmers predicted higher prices than bakers - which is evidence of wishful thinking. What's more, this bias persisted even when subjects were given incentives for accurate predictions.

This finding applies in the real world. Brad Barber and Terrance Odean have shown that retail equity investors under-perform the market around the world in part because they tend to hold on too much to losing stocks and thus suffer adverse momentum effects; this is the disposition (pdf) effect. I suspect that one reason they do this is wishful thinking: the mere fact that they hold a share makes them unduly optimistic about it. (This is closely related to the endowment effect).

If wishful thinking exists even when people have material incentives to avoid it, it is even more likely to exist where they don't. I suspect, therefore, that it is common in politics. Political partisans are prone to over-estimate the beneficial effects of their party's policies and to under-rate the sheer difficulty of manipulating complex emergent processes such as society or the economy for the better.

Maybe, therefore, wishful thinking is more widespread than David suggests.

And in some ways, this is a good thing. In Human Inference, Richard Nisbett and Lee Ross wrote:

We probably would have few novelists, actors or scientists if all potential aspirants to these careers took action based on a normatively justifiable probability of success. We might also have few new products, new medical procedures, new political movements or new scientific theories.

And Donald Davidson has written:

Both self-deception and wishful thinking are often benign. It is neither surprising nor on the whole bad that people think better of their friends and families than a clear-eyed survey of the evidence would justify. Learning is probably more often encouraged than not by parents and teachers who overrate the intelligence of their wards. Spouses often keep things on an even keel by ignoring or overlooking the lipstick on the collar. (Elster (ed, The Multiple Self p86)

And here's the point. Because wishful thinking (like overconfidence) often has beneficial effects the learning and adverse feedback that would cause us to correct our errors is absent. Cognitive biases persist because they are not selected against, and might be selected for.

I say all this for a reason. It's very easy for those of us who write about cognitive biases to commit the Homer Simpson error: "Everyone is stupid except for me." In fact, we might in some respects be as irrational as everyone else.

Comments

[Ari Andricopoulos]

I remember reading somewhere that only slightly depressed people are realistic about the prospects of success. Very depressed people underestimate and median minded people overestimate.

As you say, the reason we need it is because if we knew the true probabilities of success we'd never bother to try anything risky.

Maybe it's to compensate for the fact that failure makes one more unhappy than success makes one happy. So one has to overestimate the probability of success to take rational decisions.

[bdbd]

I'm reminded of some lines from Stoppard's Jumpers:

“The National Gallery is a monument to irrationality! Every concert hall is a monument to irrationality!—and so is a nicely kept garden, or a lover’s favour, or a home for stray dogs! .... if rationality were the criterion for things being allowed to exist, the world would be one gigantic field of soya beans!”

[Luis Enrique]

bdbd - but also David Hume: “Reason is,
and ought only to be, the slave of the passions.” - which is also the position of mainstream economics: http://press.princeton.edu/chapters/s8902.pdf

[BCFG]

I think the opposite, I think we are taught to expect less, to be pessimistic, to be cynical. Don't expect much in life and you won't be disappointed. There is nothing you, as in individual, can do to change anything. Know your place!

I think we have the opposite of wishful thinking.

I think people over-estimate the problems in making society better so don't bother trying.

This is one reason why the status quo reigns supreme.

basically you are 100% and 360 degrees wrong!

But what can you expect from someone who takes Aaronovitch as an authority?

[Metatone]

I have to agree with BCFG.

The post-Hayek consensus on information etc. is all about denying the possibility of collective action strategies. This despite thousands of years of the human race making progress through said strategies...

[aragon]

It seems my work is done. The Labour party know where to find me, but I don't expect to hear from them.

Making changes in a complex environment, not my problem any more. Others can screw up, without any help from me. They have already proved that.

https://en.wikipedia.org/wiki/Expectancy_theory

Expectancy theory when the valance is zero or negative.

Sometimes other people really are that stupid.

[chris]

@BCFG - you've highlighted a paradox. We have wishful thinking/overconfidence about individual actions, but pessimism about collective ones. Whether that pessimism is rational or not is hard to tell.
@ Metatone - I disagree. For me, Hayek's point about the impossibility of centralized knowledge is a very strong argument for democratic collective action - to eradicate central planners from companies and state organizations and replace them with decentralized collective control. Granted, Hayek hasn't been used in this way - but that doesn't mean he can't be.

[BCFG]

Ok, maybe I have highlighted a paradox or a contradiction but if you are discussing one then you can't leave out the other can you.

But I would argue that we don't really have a paradox here but a fact that the lower down the social scale you go the more people are inclined to view their opinions as being less worthwhile than those at the top and more importantly I claim this hierarchy of worthiness is generally accepted by the bulk of the population and pushed hard by the lackeys of the status quo. And in some cases the arch apologists like Aaronovitch claim the exact opposite is the reality or like to overstate the case.

I suspect your view of the world stems from the middle class and social chauvinistic world you appear to inhabit.

On the impossibility of centralised knowledge, it only looks impossible if you believe in absolute truths and when you do this, knowledge, whether 'centralised' or 'decentralised' becomes impossible.

The correct way would be to say that there are problems associated with cenralisation that don't occur with decentralisation but also centralisation of knowledge can bring into one place more data and this increased amount of data can throw more light on any given topic. So the exact relation between central authority and decentralized authority is still an open question.

I have read with interest the problems encountered by the Mondragon Co-operatives and how they have reacted to this centralisation/decentralisation problem. A worthy topic for research for anyone interested in different forms of business models.

I think it is in the spirit of this article for your readers to reject the idea of the impossibility of centralised knowledge.

[Steven Clarke]

@BCFG I agree that there is a learned helplessness lower down the scale. I'm thinking of my traditional working class grandparents. "Mustn't grumble", "Things could be worse", "There's always someone got it worse than you". I admire the stoic sentiment, but think it could be challenged more.

I quibble with your geometry. If you're 360 degrees wrong, surely you're right?

@Metatone Read Mancur Olson and subsequent authors on the problems of collective action. Collective action ultimately rests on the behaviour and incentives of the constituent individuals. If there aren't incentives and mechanisms to align the individual with the group goal, collective action is hard. This should be most peoples' experience: it's a lot easier to act by yourself or in small groups; a lot harder to get a big group to act in a certain way.

Which leads to...

@chris ...because of this, there is good reason to be skeptical of group action. And support for football teams, or the messianic fervour around Corbyn suggest there is a lot of wishful thinking for groups.

[BCFG]

"I quibble with your geometry. If you're 360 degrees wrong, surely you're right?"

I love you for that!

[Steven Clarke]

@BCFG Thanks!

If you're interested in real life use of centralised knowledge, I recommend reading 'Cybernetic Revolutionaries'. It's about the creation of a cybernetic computer network in Allende's Chile - which was to bring a lot of economic data about production into a central office for decision making by the President and others.

It never really got of the ground before his government was overthrown, but the communication network set up did help co-ordinate supplies during right-wing backed strikes.

[BCFG]

"the messianic fervour around Corbyn suggest there is a lot of wishful thinking for groups."

But I hate you for this. Corbyn represent the antitheses of the messiah complex. Under Blair we were told to have an almost fanatical belief in the power of the wealth creators and we are conditioned to view these people as messiah's. Trust in them and all your problems would go away.

So in the Public Sector a culture of managerialism came in, the belief that attracting the 'best' managerial talent would drag the Public Sector to higher standards. The end result of all that is mass cuts to public services with no decrease in taxation, hence a massive drop in value for money delivered by the publci sector. Has this reduced the faith in the power of managerialism? No, because these messiahs are above criticism, emprical evidence is not applied to these people, after all how can you apply empirical evidence to a messiah?

So everyone who apologies for this system ultimately will always, and I mean always resort to the argument of last resort, these people are just better than you!

[Steven Clarke]

@BCFG Oh well, you win some, you lose some.

I think there is a lot of messianic fervour around Corbyn (just as there was around Blair). I suspect he wants us to believe that only if we put our faith in a group of right-thinking people around the levers of government, all will be well.

How about this? The world's a complicated place where no single set of principles will work everywhere, all the time. Putting all your faith in managerialism/the State/[insert ideology here] is just going to leave you disappointed.

[aragon]

https://en.wikipedia.org/wiki/Groupthink

Type I: Overestimations of the group — its power and morality

Illusions of invulnerability creating excessive optimism and encouraging risk taking.
Unquestioned belief in the morality of the group, causing members to ignore the consequences of their actions.

Type II: Closed-mindedness

Rationalizing warnings that might challenge the group's assumptions.
Stereotyping those who are opposed to the group as weak, evil, biased, spiteful, impotent, or stupid.

Type III: Pressures toward uniformity

Self-censorship of ideas that deviate from the apparent group consensus.
Illusions of unanimity among group members, silence is viewed as agreement.
Direct pressure to conform placed on any member who questions the group, couched in terms of "disloyalty"
Mindguards— self-appointed members who shield the group from dissenting information.

[Igor Belanov]

I think there's a need to differentiate 'messianic fervour' from enthusiastic support. Corbyn's popularity has stemmed from the fact that his integrity and seeming lack of personal ambition contrast starkly with the kind of careerist conformity of his rivals, and his stress on party democracy and member's involvement has encouraged less of a cult of personality. He has also been a marked contrast from flawed populist 'demagogues' like Galloway or Sheridan.

[Dave Timoney]

What is hilarious (or touchingly sad, I can't judge the context beyond the Times paywall) about Aaronivitch's position was his own anxiety to believe the fantasies and frauds of the early 00s.

Some of Corbyn's popularity (like Miliband's before him) derives from his evident limitations as a thinker and political strategist. It is his very naivety that people find attractive, though they gussy it up as integrity and decency.

This may be an indulgent reaction to what has gone before, but there's no denying that it is a reaction to a political landscape that the likes of Aaronovitch did so much to cultivate.

[gastro george]

@Steven Clarke: "I suspect he wants us to believe that only if we put our faith in a group of right-thinking people around the levers of government, all will be well."

I wouldn't think Corbyn thinks that. I'd imagine that he and his supporters think the reverse, that "faith in a group of right-thinking people" has been proven to be wrong, and that there is an alternative. The wishful thinking is more a hope, and hope of an alternative is very seductive. But I wouldn't think that Corbyn thinks that he has all of the answers right now. But he offers a new direction to travel in.

[jdc325]

"I say all this for a reason. It's very easy for those of us who write about cognitive biases to commit the Homer Simpson error: "Everyone is stupid except for me." In fact, we might in some respects be as irrational as everyone else."

Quite. I used to think I was rational and sceptical, but every time I read a pop psych book about irrationality I managed to find a new flaw in my thinking. Perhaps not surprising when there's a cognitive bias of failing to compensate for one’s own cognitive biases: https://jdc325.wordpress.com/2010/12/23/beware-the-bias-blind-spot/