The illusion of knowledge

More information does not mean that we make better decisions, or even good ones.

Some laboratory evidence for this comes from some experiments (pdf) at Princeton. Alexander Todorov and colleagues asked subjects to predict the results of basketball games. Half of them were told the teams’ records that season and the half-time score in the match concerned. The other half were also told the names of the teams. Those who were told the names made worse predictions but with more confidence than those who weren’t. “More knowledge can decrease accuracy and simultaneously increase prediction confidence” they concluded.

A couple of examples from my day job corroborate this. Economists at the University of Mannheim show that even sophisticated investors buy expensive but poorly performing actively-managed funds. And economists at the University of Maryland show that when informed investors are given advice from a portfolio optimization tool they trade more often without improving investment performance.

You might think these examples are small beer. But a similar thing might have lain behind some of the most catastrophic decisions of recent years. If the government had had zero information about Iraq in 2003 rather than the military intelligence it actually had, it might not have gone to war. And if banks had known nothing they might have held fewer credit derivatives in 2007 than they were led to own by AAA credit ratings and risk management systems which told them that big losses would be 25 standard deviation events.

There are at least two mechanisms whereby more information leads to worse decisions.

One is that it encourages overconfidence, thereby emboldening people to do things they wouldn’t otherwise do, or to place bigger bets than they otherwise would. They fail to see that even good knowledge still leaves a lot of unavoidable uncertainty about the future.

A second is that information distracts people from using simple but effective rules of thumb of the sort described (pdf) by Gerd Gigerenzer (pdf). Simple rules such as “invest in passive funds” or “don’t start wars” work reasonably well: they at least protect us from terrible errors. Knowledge – and especially the illusion of knowledge – can lead us to think we can better than this, even if we cannot.

All this might seem abstruse. But in fact it bears upon two of this week’s big issues.

One is the idea of ranking universities by graduates’ earnings. Some people responded to my scepticism about this exercise by saying that if the rankings are prepared carefully they’ll be better than nothing. Weak – and necessarily out-of-date – information might however distract students from using effective heuristics such as “go to the university with the best reputation you can given your grades”.

The other issue is the Skripal affair. People on all sides seem wholly confident about what happened and what the UK should do on the basis of information that seems to me to be scant at best. The heuristic “if you don’t know, shut up” goes unheeded. But then, perhaps in this case the evidence is not the point.  

Comments

[Jim]

Isn't this pretty much the entire reason socialism is a crock of sh*te?

If huge amounts of decision making is wrong, it hardly means a way of organising society that is predicated on one central authority deciding what everyone should do or have is going to end well, does it?

Whereas when everyone makes their own individual decisions, yes a lot will be wrong, but the people making the wrong decision have to bear the consequences of their own poor choices, but as a society we get to see which decisions are wrong and which ones are right a lot quicker and with less societal upheaval than under socialism.

[e]

I'm sorry but it is not the case that people on all sides are wholly confident re: the Skripal affair. More precisely one group, the grouping led by JC, is acknowledging known unknowns, which is not to equate a lack of confidence re the details with a lack of confidence re the way forward, it's to deplore our current drift towards an illusion of mighty isolation.

[kernel]

Bad military intelligence did NOT lead the US & UK to invade Iraq. Rather, the choice to invade Iraq led the US to invent bad intel as excuses/cover/propaganda. My impression is that the UK went along with it for political reasons, and played the same game with the intel.

But this bad example doesn't really undermine your thesis (more info is often worse). Seems related to the Dunning-Kruger, on a micro level.

[Dipper]

"The other issue is the Skripal affair. People on all sides seem wholly confident about what happened and what the UK should do on the basis of information that seems to me to be scant at best."

This isn't a game. Two people who should have been safe are now in prison dangerously ill poisoned with a nerve agent. One country stands out as having the motive and the means. No-one else has either.

Neither is this a criminal court case. Standing on pedantic points of what exactly constitutes prove and what doesn't whilst people are being poisoned in the streets isn't principle or leadership, it is at best stupidity and at worse cowardice. I sincerely hope we never have to rely on JC and his disciples to stand up to protect the citizens of this country, as on past form he will find excuse after excuse to avoid standing up for UK citizens.

[Gipper]

>>>One country stands out as having the motive and the means. No-one else has either.

Now is that a fact?

Your mastery of world affairs and logic extends to the assumption that if a murder is attempted on a person, a country must have done it.

This is not a game, indeed.

[rogerh]

I think kernel touches on the problem. In a mature market investors will be scratching around for some sort of edge and providers of 'edges' will be only too pleased to provide them. Experience suggests that most of these manufactured 'edges' are phoney or of negligable use but they make copy for trade magazines and talk shows and politicians.

Chris's advice for students seems eminently sensible and we could confidently expect that university course rankings would quickly become corrupted by commercial and political interests to the point of being worse than useless.

As for Mr Skripal, having had a lucky escape from prison he might have been wiser to keep right away from his old trade. Both himself and those who encouraged him to continue must bear some blame for what happened.

[Dipper]

@ Gipper - You are a Russian Bot and I claim my £5.

[billb]

@Dipper - You are the Russian Bot

[D]

@Dipper - I'd bet quite a lot that it was Russia. And agree it's not a criminal case in court, so we probably have to behave as if it was them (although that doesn't necessarilly mean escalating things further than we have already). But the idea that no one else has either the means or motive is just silly. For one, the type of nerve agent was developed by Russia, that doesn't mean no one else can make it. And how do you know there aren't other people or countries that have the motive?

[Brian]

1. Military intelligence is an oxymoron.
2. I’d rather be treated by a knowledgeable medical doctor than a bloke from down the pub.

[Robert Mitchell]

@ Jim

>> when everyone makes their own individual decisions, yes a lot will be wrong, but the people making the wrong decision have to bear the consequences of their own poor choices >>

What consequences did AIG bear? UBS? UBS created more money in the run-up to the 2007-2008 crisis than they wrote down in 2008.

The clear implication is that, as Fischer Black noted in his 1986 essay "Noise":

>> we might define an efficient market as one in which price is within a factor of 2 of value, i.e., the price is more than half of value and less than twice value.11 The factor of 2 is arbitrary, of course. Intuitively, though, it seems reasonable to me, in the light of sources of uncertainty about value and the strength of the forces tending to cause price to return to value. By this definition, I think almost all markets are efficient almost all of the time. “Almost all” means at least 90%. >>

https://onlinelibrary.wiley.com/doi/full/10.1111/j.1540-6261.1986.tb04513.x

In other words, prices are pretty arbitrary. If oil is $100 per barrel, noise can make it $50 per barrel or $200 per barrel. That range encompasses 300% inflation that can be due to noise alone. Thus oil prices are pretty arbitrary, and the stories we tell about inflation resulting from money creation are just knowledge leading public policy to make wrong decisions about creating money for social spending.

[James]

"Weak – and necessarily out-of-date – information might however distract students from using effective heuristics such as “go to the university with the best reputation you can given your grades”."

I take issue with this - "reputation" often gathered over at least 50 years, sometimes hundreds is surely a more out of date heuristic than a "what are graduates earning 5 years after graduation?" type heuristic.

This isn't to say that either is perfect but to what extent is someone who is invested enough in such an important decision as which university/course they attend going to be "distracted" by more information? My personal opinion is that this imperfect information would allow students who value high earning potential to at least get a better view of which parts of a university's reputation are due to their cherished earning potential and which due to other factors.

[Blissex]

«I sincerely hope we never have to rely on JC and his disciples to stand up to protect the citizens of this country, as on past form he will find excuse after excuse to avoid standing up for UK citizens.»

If I understand well Gavin Williamson, the minister of defense, has boasted to widespread applause that he runs a large network of death squads, and that in regular meetings he has judged and sentenced to death dozens to hundreds of UK citizens as he regards them as potential enemies of the government, and has has tasked the UK death squads with liquidating them wherever they are.

http://www.dailymail.co.uk/news/article-5153613/Gavin-Williamson-Brits-fighting-be.html
«Earlier this year, the Mail revealed that RAF pilots had been secretly assassinating British jihadists in Syria and Iraq, using drones and fighter jets to work through a ‘kill list’ of targets»

[Dipper]

@ Blissex yes.But the fact that Gavin Williamson is revealing himself to be an overpromoted schoolboy with does not detract from JC's moral cowardice.

[Nanikore]

I am not sure that the investment banking analogy is a good one. Much of these decisions were based on mathematical models - which is not the same thing as information or real knowledge.

[Padraig]

On Skripal, I'm going to use an old heuristic - Occam's razor. If I have to choose between believing a) that Putin ordered a hit, as he has done before, on a person he's obliquely referred to many times as a traitor, and where Putin and Russian media have often talked about how dangerous it is to be a traitor, and also the nerve agent was Novichok, and b) that it was a false flag by someone to do something for some reason, well I'm going to go with a.

I'm leftist, and Corbyn is freaking me out right now. He's reacting as though there's talk of war with Russia, when no one in their right mind (and also no one in power) thinks that's a good idea. But there's a lot to be done short of war, starting with going after the money held and laundered in London by Putin cronies.

[Ben Philliskirk]

@ Padraig

"I'm leftist, and Corbyn is freaking me out right now."

"But there's a lot to be done short of war, starting with going after the money held and laundered in London by Putin cronies."

Yet that is exactly what Corbyn is suggesting, and what the Tories have dragged their heels on for years.

[Dipper]

@ Padraig (and others)

"starting with going after the money held and laundered in London by Putin cronies"

good luck with this.

There is extensive anti money Laundering law already in place. This includes specific individual responsibility for everyone in banking to check and inform. There are extensive Know-Your-Customer processes in every bank that are heavily overseen by regulators. If, however, someone has successfully laundered their money outside the UK then there isn't much legally you can do to stop it coming in (apart form capital controls).

[Dipper]

@ Padraig "I'm leftist, and Corbyn is freaking me out right now"

Well yes. He is an empty-headed space where a politician should be. He makes vague noises but never makes an firm commitments.

Hence The Battle For Corbyn's Brain. Labour is getting very ugly because everyone knows that the public discontent with the Tories and austerity is getting to the point where Corbyn could end up Prime Minister, and everyone knows that whoever is pulling Corbyn's strings will get to decide what kind of government that will be. Milne and his fellow tankies are clearing the way for hardline marxism. McDonnell senses that this is the one chance he may get to hold the leavers of power and is desperate for The Tankies not to screw this up so is doing his best Kindly Old Uncle John act. The closer we get to the empty vessel that is Corbyn gaining power, the more desperate will be the battler to occupy Corbyn's brain.

[Nanikore]

Dipper, I think you are right. What might disturb many is that figures who are associated with New Labour have managed to position themselves close to McDonnell and are formulating his policy platform. On the economy at least, a Corbyn Government may end up not looking too different to a Blair one. It seems that after the financial crisis, Brown's defeat and Brexit, lessons are still not being learned. We could have even worse consequences than Brexit to come.

[Roger]

Your post just argues against making a decision based upon erroneous data or cognitive biases. Nothing new there.

You then try to use it to imply the income measures which are not yet developed but which you assume will be out of date and naive will be better than whatever the heck "university reputation" measures today (probably a function of elitist acceptance, arbitrarily high tuition and the number of sushi restaurants in the quad.

Seriously you are basically making an argument for reducing the information around one of the most important and costly decisions of a person’s life.