Costs of overconfidence

Ben Chu is absolutely right to bemoan the damaging effects of overconfidence. I’d just add a few things.

One is that overconfidence can be a form of strategic self-deception. A new paper by Peter Schwardmann and Joel van der Weele shows this. They got subjects to do intelligence tests and then selected some at random and told them that they could earn more money if they could convince other subjects that they had performed well on the tests. They found that the selected subjects were even more overconfident about their performance than non-selected ones.

This is evidence for strategic overconfidence: people deceive themselves in order to better deceive others. I suspect this – or perhaps just ordinary deception – is common in debating societies and newspaper opinion pieces.

And overconfidence – strategic or not - succeeds. Schwardmann and van der Weele also show that the overconfident subjects were rated as smarter by evaluators. This corroborates a finding by Cameron Anderson and Sebastien Brion, that overconfidence is wrongly perceived as a sign of actual competence. This is one reason why job interviews are a bad way of selecting people (pdf).

Schwardmann and van der Weele say:

Overconfidence is likely to be more pronounced in settings where its strategic value is highest ie where measures of true ability are noisy, job competition is high and persuasion is an important part of success. Accordingly, we would expect overconfidence to be rife amongst high level professionals in business, finance, politics and law.

We might add bosses and journalists to this list.

This is not to say that strategic self-deception is all there is: Schwardmann and van der Weele estimate its only one-third of the story. People are also overconfident even when they lack extrinsic incentives to be so.

I fear that this overconfidence has systematically distortionary effects, in at least three ways:

 - As Ben says, it perpetuates inequality. People from posh or rich backgrounds are more likely to be overconfident than others, and hence more likely to get jobs - because they are more likely to push themselves forward as well as more likely to be selected when they do. Trump’s discovery that being president isn't as easy as he thought it would be echoes Toby Young’s finding that running a free school is harder work than thought, and David Cameron’s discovery that he wasn’t in fact “rather good” at being prime minister.

 - It’s not just political candidates who are overconfident. So too are some voters. Erik Snowberg and Pietro Ortoleva show that this contributes (pdf) to extremism.

 - Overconfident people don’t just misperceive their own abilities. They also misperceive the world. They under-estimate the complexity of social phenomena and the boundedness of their rationality and knowledge. This leads to terrible decisions, such as Blair’s war in Iraq or Cameron’s belief that he could win the EU referendum.

Theresa May’s endless talk of “strong and stable” leadership fits this pattern. Of course, it poses the question: is she really strong or is this yet another example of overconfidence? Dogmatism and inflexibility are not the same as strength. Often, people boast of qualities they are unsure of: truly intelligent people don’t brag about their intellect, nor kind ones of their kindness.

Ms May hopes the voters won’t ask this question. And the evidence suggests that, sadly, she is right to do so.

Comments

[Luis Enrique]

do give it a rest. I am busy trying to develop overconfidence to further my career prospects and you are really busting my vibe.

[Steven Clarke]

So this is a subset of those cases where individually rational behaviour is collectively irrational.

[Eminent emigrant]

Worth reading Toby Young's comment/rebuttal at the bottom of that article. Maybe the journalist was over confident :)

What you day here ties in rather scarily with the Dunning-Kruger effect (the name of rich I can never remember!).

[JohnK]

Dunning-Kruger is a good framework to look at this (competence is inversely related to self-judgement of competence), as is the Peter Principle (people are promoted to a level one above their ability to perform competently). Theresa May seems to show a combination of both problems.

[Ralph Musgrave]

Chris says “people deceive themselves in order to better deceive others..”. That’s 90% right. What’s missing is the point that to make a point in as persuasive a way as possible, you must seriously believe in it yourself (even if your intellect is telling you the point is BS).

Actors make that point when they say (apparently) that to act out a part as well as possible, the actor must actually become the character they are acting.

[Rick Shapiro]

Overconfidence, like tribalism, is a universal trait in humans for clear evolutionary reasons. Accurate internal representation of the world is not the utility function of the brain. The utility function of the brain, like that of every other element of the phenotype, is maximization of descendants. In many circumstances this entails accurate representation of the world; but in others, particularly in social circumstances, it entails internal mis-representation.

[Keith]

Cameron failed over the EU as his overconfidence arose from attributing his successes in life to his personal qualities rather than class privilege. When his con job of trying to circumvent the eurosceptics inside and outside the Tory party came up against the media barons the media won. They told people to vote leave and they did rather than listen to Mr Cameron and his then chancellor. hence showing what happens when members of the committee set up to advance the interests of the ruling class try to buck their paymasters. Cameron got the Saddam Hussein treatment for getting above himself. In his case the slap was mild.