Costs of under-confidence

In the day job, I partially endorse Daniel Kahneman's claim that overconfidence can be a damaging cognitive bias.

However, as Niels Bohr said, the opposite of a great truth is another great truth. The other night, I was reminded that under-confidence also has costs. In our pub quiz one of my team-mates suggested several answers but with little confidence, causing our captain to choose other answers. However, she was right every time and our captain wrong. The upshot was that we slumped to an abject defeat to the bottom team in the league. Had Linda been more confident, we'd have won.

The point generalizes. The old saying, "faint heart never won fair lady" reminds us of the cost of under-confidence.

In Human Inference, Richard Nisbet and Lee Ross wrote:

We probably would have few novelists, actors or scientists if all potential aspirants to these careers took action based on a normatively justifiable probability of success. We might also have few new products, new medical procedures, new political movements or new scientific theories.

They might have added that we would have even fewer if people had been under-confident. Rick's right that overconfidence can generate too much self-employment. But under-confidence gives us too little*.

This reminds me that one of the under-rated skills in life is the ability to know one's skill. This might well be one of the keys to success of Berkshire Hathaway. Charlie Munger has said of himself and Warren Buffett that: "We know the edge of our competency better than most.” There are two elements to this. One is knowing what you don't know - not being overconfident. The other is knowing what you do know - in Buffett and Munger's case, that it's a good idea to bet against beta, especially when you have (pdf) a low cost of capital.

This is perhaps especially clear in sport. In most sports, one often has a choice between a risky option and a safe one and the wrong choice has costs. The overconfident batsman tries risky shots too much and loses his wicket; the overconfident footballer tries the risky pass and loses possession. But the under-confident batsman plays too defensively and the under-confident footballer passes sideways too much - as Manyoo fans might have noticed this season.

It's in this context that Mesut Ozil stands out. One of his great abilities is to keep the ball even in attacking areas: he's completed 87.9% of passes in the Premier League this season, a ratio usually associated with defensive players who play safer balls; Wayne Rooney's ratio, for example, is only 81.4%. A big reason for this is that he knows his skill - when to play safe and when to play the attacking ball. As Arsene says, he's "like a musician who always plays the right note, and at the right moment."

What matters is that one's confidence be properly calibrated - neither too high, not too low. Very few of us achieve this.

* I'm not sure the distinction between self-employment, innovation and entrepreneurship is a sharp one. Spotting that a town could use another tea shop is a sort of innovation.

Comments

[Blissex]

«What matters is that one's confidence be properly calibrated - neither too high, not too low. Very few of us achieve this.»

Amazing conclusion! If we knew *in advance* what is the right level of confidence in our guesses it would be really golden! :-)

«under-confidence also has costs. In our pub quiz one of my team-mates suggested several answers but with little confidence, causing our captain to choose other answers. However, she was right every time and our captain wrong.»

I suspect that you may be confusing between assertiveness and confidence here. Perhaps she was quite confident, but not very assertive, so her confident guess was dismissed.

[Blissex]

«was quite confident, but not very assertive»

BTW there is a different topic here: the goal of a pub quiz is not to win, but to have fun, as a social occasion. Someone who is knowledgeable and usually right is usually a cause of irritation in british culture, and "know-it-all" or "egghead" are not terms of endearment. It is also particularly unpopular to be confident or even worse assertive with one's social superiors or equals (it is plenty acceptable with inferiors of course). So perhaps your team-mate was embarrassed.

Perhaps with your Oxbridge background (even if IIRC far from the that of the Bullingdon Club, but rather more "swotty" perhaps) that's something that has not occurred to you...

[Luis Enrique]

this is all very true - although if confidence means subjective probability of success, then attitude towards failure also important (i.e. willingness to have a go for any given prob of success).

[Phil Beesley]

@Luis Enrique: I used to play pub quiz trivia machines with a mate. We made more money from those machines than anyone else locally, but it was just beer subsidy.

We trusted one another to be "experts" in topics. My mate overrode my choice occasionally, perhaps wrongly, perhaps rightly. I did the same to him. We were equally assertive.

The word "trust" is significant. I trusted my mate.

I know that I cannot fully understand climate change science. If I spent a couple of years studying it, I could comprehend the basics. Instead I think and listen for a few hours and take in much.

And then I have to "trust" climate scientists? I can trust the science bit of climate science but I don't trust the economics.

[Jason Smith]

Calibrated confidence is a good way to put it -- but that calibration depends critically on what society provides in the case of failure. If failure means becoming destitute on the streets, over-confidence should be calibrated down.

I used ET Jaynes concept of "dither" -- exploring the economic state space -- as another way of looking at it. You need confidence to explore new parts of the economic state space which could explain where growth (personal as well as aggregate) comes from:

http://informationtransfereconomics.blogspot.com/2015/04/thinking-positive-is-thinking-different.html

[SeanV]

The Yale link appears to be broken.

[rogerh]

Can some of this effect (and its opposite) be seen in recruitment adverts. Some I sampled seem quite gentle and overtly welcoming whilst some dictate extensive experience of this that & the other. Can it be the advert is tuned like a dog whistle to the kind of people you want to recruit.

[An Alien Visitor]

My solution to this problem is to abolish job interviews and instead put all the applicants names in a hat and draw it out at random.

The applicants being those whose CV's pass the requirement for the job of course.

[Fab L]

Two more aspects:
Interestingly enough, the "optimal" level of confidence might differ from the one warranted by the facts, and to both sides, depending on the situation (and in particular on the cost of being wrong).
When participating in a pub quiz on a topic one is an expert on, one might be well advised to be less confident (or assertive, as someone suggested) than the facts warrant, to not annoy one's teammates too much.
When deciding whether to approach that fair lady, lads can be way too confident, as the cost of rejection pales compared to the bliss of success. (Though, taking the asymmetry of rewards into account, maybe the level of confidence is warranted by the facts?)
The other aspect to note is that the optimal level for the individual might be quite different from the socially optimal level. For example, most entrepreneurs fail, so they might have been better off less confident. However, the benefits of successful innovation to society are so large, that a considerable number of bankrupt ex-entrepreneurs might be a small price to pay for individual over-confidence.

[Strategist]

>>> I know that I cannot fully understand climate change science. If I spent a couple of years studying it, I could comprehend the basics.

I find this this a mystifyingly prevalent form of underconfidence. The 'basics' can be understood with sub-GCSE physics in a minute.

One very large sphere; one external source of energy. Amount of heat retained dependent on composition of the atmosphere. Change the composition, change the average temperature of the sphere.

[Sven]

"Had Linda been more confident, we'd have won" Really? I think it's one of the big problems for the quiet introvert types - and their tendency to be under-confident - that the common perception is it's their fault for not speaking up. I believe this should be turned around: "Had the captain been more perceptive and trusting we'd have won." It's the captain's job to hear the soft-spoken truth.

[Otavio Zabaleta]

Well, I like to say "Be honest inside, admit doubt and fear but, once you came to a conclusion that needs to be pointed, look confident." Many times people mistake lack of confidence (or honesty about not being 100% sure) for lack of knowledge/competence/etc. So when communicating something for those people, you must be confident, or they'll always disregard you. Maybe they're not stupid, it's just that they feel more confident about their notions the more they gave it thought, and don't realise your Linda's process is different.

[Otavio Zabaleta]

Agree with @Sven, it's the leader's fault more than Linda's. How many potential is wasted just because so many people disregard whatever comes out of a timid person's mouth.

[Greg]

My daily pre-work routine involves quieting my information-hungry inner gooner (who gets disruptive if he's not fed) by opening up a few Arsenal blogs. Only after this can I move on to more stimulating, work- and idea-oriented ones like yours, which set me up for the day.

There has been a transgression. Me and my inner gooner are now both deeply disoriented. My work will suffer. Please include trigger warnings in future.

[An Alien Visitor]

Taking up Sven's point. I have sat in job interviews and I remember one occasion when someone who had worked in our section and had done an exceptional job, applied for a higher grade and in the interview they were so timid and introverted there is no way they would have got the job had we who sat in the interview didn't know them.

And we would have lost one of our best employees! Now what I don't know is if that blinded me to an even better applicant. Hence, put the names in a hat and draw them at random. Don't rely on the leaders wisdom!

[shah8]

Structural oppression based on race and gender etc is meant to encourage underconfidence.

[Phil Beesley]

@Strategist -- It depends on how you define the "basics of climate science". The atmosphere (assuming a definition) may not be a control volume and whilst it is easy to see how CO2 enters it in broad terms, CO2 also exits. Plants and trees use CO2 to grow, but eventually they die or are consumed in a way which releases CO2. Plants are also stuck down here on earth away from most of the atmosphere. So how do we remove CO2 from the upper atmosphere?

Once you read the answer to one basic question, more questions arise.

[Mike Lloyd]

@Phil Beesley. Rather than have a "basics of climate science" here, suggest you look at

http://www.realclimate.org/index.php/archives/2007/05/start-here/