The irrational virtue of stubbornness

Arsene Wenger and George Osborne don't seem to have much in common - only one of them has a qualification in economics - but they do. Both this week expressed a will to stay the course despite criticism. Osborne says that Moody's downgrade merely "redoubles" his determination to stay with his plan, and Wenger says he has never for "one second" considered resigning.

Such stubbornness is, of course, common in many walks of life. It is often the product of many cognitive biases, such as:

1. Bayesian conservatism. Both Wenger and Osborne have strong priors that they are right, and they regard evidence to the contrary as sufficiently noisy that it  can be disregarded.

2.Ego involvement.For either man to step down would be a recognition not merely that a set of intellectual propositions is wrong, but that they themselves are not the men they thought they were - not the great Chancellor or great coach.

3. Gratification of enemies. An acknowledgement of error would be a victory for things they hate - Ed Balls in Osborne's case and financial doping in Wenger's. It's one thing to admit error in oneself; it's another to surrender to a loathsome foe.

4. Wishful thinking and overconfidence. It's surprisingly easy to believe things will turn out to one's advantage, especially if you think you're right anyway. So Osbone thinks business confidence will pick up because of low interest rates, and Wenger thinks his players will improve and that he can find new players to strengthen the squad.

It seems, then, as if Wenger and Osborne have much in common. Except for one thing - Wenger is a genius and Osborne, well, might not be. The difference, I suspect, lies in point 1. Wenger's prior that he knows best is founded upon things like: the fact that he's the only coach in 130 years to have taken a team through a whole top-division season unbeaten; the fact that no team that cost less to assemble has finished above Arsenal; and that he has turned countless players from unknowns to world stars. Osborne's prior is somewhat less well-founded.

Yes, Wenger might be biased. But sometimes, cognitive biases are a good thing, as they give us the strength to stick with a correct course in the face of adversity. The author who gives up after a few publishers reject her work, the businessman who gives up after a bank turns him down or sale falls through, and the musician who gives up after a few bad gigs might be behaving rationally. But rationality is not necessarily the road to success. As Richard Nisbet and Lee Ross wrote in Human Inference: Strategies and Shortcomings of Social Judgement, one of the earliest books on cognitive biases:

We probably would have few novelists, actors or scientists if all potential aspirants to these careers took action based on a normatively justifiable probability of success. We might also have few new products, new medical procedures, new political movements or new scientific theories.

If it weren't for the benefits of the biases that generate stubbornness, then, we'd have little art, music, business or science - and we'd probably be living under a Nazi dictatorship.

We should not, then, criticize Osborne for being stubborn. Stubbornness can be a virtue. The problem instead is that he's simply wrong.

Comments

[FromArseToElbow]

Wenger's frustration in the face of press criticism stems from the assessment of performance relative to expectation - i.e. the press have fabricated an expectation (which they disingenously impute to the "fans") that Arsenal should regularly win silverware despite their resources currently pointing to an EPL finish between 4th and 6th and a CL finish between last 16 and last 8.

Osborne's problem is that the expectation in his case is of his own making: the projections for deficit reduction, the totem of the triple-A rating, and the expansionary contraction powers of austerity. In comparison to Wenger, he's had a remarkably easy ride. The worst press he's had so far has been for taxing pasties, which would be a bit like saying Wenger's worst "crime" was letting Bendtner go out on loan.

[ricketson]

I don't get the Nazi comment. Didn't the Nazis show this same type of stubbornness? My guess is that this stubbornness would also be required for the establishment of national monarchies. If it weren't for this stubbornness, I would expect our political organization to be something very local -- like a very decentralized feudal or clan structure.

[Keith]

"I don't get the Nazi comment. Didn't the Nazis show this same type of stubbornness?"

yes. In the area of Politics stubbornness is a common cause of failure. Hitler defeated himself by embracing wrong strategy and following it to the point of getting his armies in Russia destroyed. Forbidding them to surrender or negotiate after getting them stretched to the point where they could be subject to encirclement; and Stalin killed so many of his own Army and party/ state officials he handed his empire to Hitler on a plate in 1941 saved by the aforementioned mistakes of Adolf. Thatcher and the Poll Tax springs to mind. Football Managers cause little harm compared to Pols when they get divorced from reality.

[Andrew]

Muddy thinking Chris.

If Wengers' strong priors are well founded (or even, for that matter, if they aren't) then using them to outweigh new information is perfectly rational.

Conversely, sticking to beliefs DESPITE new evidence tipping the Bayesian scales would not be an advantage, but a disadvantage.

Thirdly, this constant obsession with needing overconfidence to produce successful people; maybe sometimes you do, but a decision to go into acting is actually very rarely based on overconfidence about the chance of success, and often much more to do with enjoying acting. And whether a low probability chance is worth a shot is entirely down to how you calculate those "normative" odds and payoffs, isn't it? Sometimes just having a damn good go is a payoff in itself.

All this waffle about overconfidence biases hints at an evolutionary explanation - better for a gene to have a hundred individual copies try for reproductive success and one succeed rather than none try. So individuals evolve to try, even against their rational individual interest.

However, is it really against their individual interest if their makeup includes a strong drive (ie desire) to try?

I think people who delight in finding "irrational" behavior need to remember that we all end up dead, and a rational course of action depends ENTIRELY on what the passions have dictates as goals.

Some deviations from rationality are well-documented artifacts of cognitive heuristics. Others are much more woolly.

[Andrew]

Good points about rationality as discussed in economics in these links posted on previous thread.

http://crookedtimber.org/2005/10/14/rationality-repost/

http://crookedtimber.org/2011/09/24/the-poverty-of-rationality/

[BenSix]

I hope this does not sound like damning with faint praise but the first sentence must be acknowledged as a thing of rare beauty.

[chris]

@ Andrew - of course, you're right that it's rational that strong and correct priors should outweigh weak new evidence. My point is rather that other biases might reinforce this too much - eg if someone thinks "I'm 100% right" when he should think "I'm 90% right".
It's pretty well attested in financial markets that overconfidence is both widespread and costly:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=529062
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=94143
It would surely be odd if such costly overconfidence did not exist in other spheres.

[Ding]

'Wenger's prior ?'
Don't understand the use of 'prior' in this context. Do you mean 'priority' ?

[Agog]

Of course there's the difference that while Wenger has clear targets that everyone can agree on (winning football matches, and thereby competitions), Osborne might very well be succeeding on his own terms but not on many other peoples' Surely everyone knows that Osborne is not trying to achieve some kind of utilitarian optimum, or even to maximise aggregate incomes. Even future electoral success might not be top of the list for all we know. Who can say W exactly TF is going on in the mind of a guy like that?

[Andrew]

@Chris - It is also right that strong and incorrect priors should outweigh new evidence.

Any prior may turn out to be wrong.

If you are going to criticize a prior, you must either criticize the process that led to its formation, or demonstrate internal incoherence of a set of beliefs.

Now, yes overconfidence is a documented trait in various spheres. Whether it is costly or not (and therefore whether it is really overconfidence) depends entirely on how you price the outcomes, as I alluded to in my earlier comment about evolution.

In financial markets the costs are not borne entirely by the agents.

What does the evidence say about simpler natural models of long run statistical decision making, such as high level professional poker?

[Andrew]

And yes, that first sentence was very good!