Experts & the demand for certainty

It’s a bad day for experts. The Times complains that economic forecasters are as blind as ancient soothsayers, whilst proof that Colin Stagg was innocent discredits Paul Britton’s expertise as a forensic pyschologist.
To point out that experts are wrong, however, is to misunderstand the purpose of them. Their function is not to provide knowledge, and still less clear thinking. Instead, it is to provide certainty. People hate dissonance, doubt and uncertainty. Experts help dispel these. So, Paul Britton’s function was to tell the police that they had the right man, whilst economic forecasters’ job is to provide an impression that the future is knowable; no-one wants to hear about standard errors, parameter uncertainty or the Lucas critique.
What’s so pernicious here, though, is that people have ways of achieving an illusory certainty anyway. As Sir Harry Ognall - the judge who acquitted Stagg - says: “The police closed their minds to any other possibility than that of his guilt.”
There are several ways they got these closed minds. All have analogues in corporate planning and financial trading.
1. The confirmation bias. Having acquired the belief that Stagg was guilty - he fitted the profile, was on the scene and a bit of a weirdo - subsequent evidence was interpreted as corroborating this. So, the fact that he looked shifty in interview was seen as evidence of guilt, not as the sign of an innocent man nervous of being fitted up.
A similar thing happens in economic forecasting. If you thought last week that we’re heading for a very deep recession, you put great weight on Wednesdays’s jobless numbers and find ways of dismissing yesterday’s retail sales figures. If you thought the recession would be mild, you do the opposite.
2. The halo effect. In their book, Mistakes were Made, Carol Tavris and Elliot Aronson describe how policemen believe “I couldn’t have been wrong because I’m a good guy.” Even if we grant the premise, the error here lies in believing that good qualities - moral rectitude and cognitive skills - must be correlated. They are not. Of course, coppers are not unique in thinking this.
3. Groupthink. If our colleagues agree with us, our confidence in our judgment rises, especially if we like them. 
This error arises in part because we fail to see that correlated data points add little to certainty. If our colleagues have the same training and evidence as us, and are also prone to groupthink, their beliefs will be correlated with ours, and so will not be new evidence - no more than a second copy of the Daily Mail corroborates the stories made in the first. But we interpret them as if they are.
4. Ego-involvement. Admitting that we are wrong means more than just fessing up to narrow technical error. We interpret it as a blow to our ego - a sign that we are not the infallible, uber-competent professionals we think. We’ll do anything to squirm out of facing this. Hence the failure of the police, until yesterday, to apologize to Colin Stagg, and the failure of many bank bosses, Tom McKillop excepted, to apologize for their errors. 
And herein lies the purpose of experts. It’s to reinforce these mechanisms, to help people avoid the uncomfortable facts that the world is uncertain, that mistakes are inevitable, and that we are not as in control of things as we think.
Blaming experts for being wrong is like complaining that the economy is not yellow. It’s a category error so howling as to be nonsensical.

Comments

[dearieme]

It's certainlv a category error to talk of "experts", undifferentiated.

[Iain Coleman]

Given that these are basic facts about human psychology, the question arises of what to do about them. We can't change them: we need structures that counter them.

In science we have peer review: an imperfect process, but one which does work reasonably effectively against the psychology that you cite. It does so by introducing new players whose incentives are explicitly to attack unwarranted assumptions and unsound reasoning.

It's not obvious how something similar could be implemented by the police, but perhaps these are the sorts of lines they should be thinking along.

[Bob B]

Let's face it, there's no shortage of alternative sources of quality commentary on economic affairs in the financial press or from independent think-tanks and when economists - unlike finance market professionals - are not noted for herd-like propensities.

Indeed, the more frequent complaint is that economists are apt to disagree, which makes it the more significant when there's a broad consensus among most economists on an issue.

IMO we ought to worry rather more about the conspicuous extent of economic illiteracy, especially in the political class.

[Morgan]

The four categories also explain why tabloid editors had Stagg hung, drawn and quartered before he even came to trial.

[Morgan]

The four categories also explain why tabloid editors had Stagg hung, drawn and quartered before he even came to trial.

[ian]

Consider the words of Lord Denning in the case of the Birmingham Six

"If the six men win, it will mean that the police were guilty of perjury, that they were guilty of violence and threats, and that the convictions were erroneous. This is such an appalling vista that every person in the land would say: 'It cannot be right that these actions should go any further.’ It is better to keep innocent men in prison, than to let them go free and bring the system into disrepute."

[Dipper]

J K Galbraith to the rescue:

"Faced with the choice between changing one's mind and proving there is no need to do so, almost everyone gets busy on the proof"

"In any great organization it is far, far safer to be wrong with the majority than to be right alone."

"The only function of economic forecasting is to make astrology look respectable "

[ortega]

Their function is not to provide knowledge, and still less clear thinking. Instead, it is to provide certainty. People hate dissonance, doubt and uncertainty. Experts help dispel these.

Really? I'm not that sure, because, speaking only of economics, you have as much opinions as experts, holding each possible option and its contrary.

[Bob B]

"The only function of economic forecasting is to make astrology look respectable "

That is an unbelievably silly comment.

Governments of any colour inevitably need to make forecasts of tax revenues and the consequences of policy changes. Central banks with a remit of inflation targeting have to assess the downstream effects on inflation of changing interest rates.

[Tom James]

@ dearieme "It's certainlv a category error to talk of "experts", undifferentiated."

Exactly - in the context of political and social topics, yes: but when it comes to physics, medicine, engineering, or plumbing you can talk of experts.

@ Bob B "IMO we ought to worry rather more about the conspicuous extent of economic illiteracy, especially in the political class."

Add to that basic knowledge of the existence of cognitive biases, empirical evidence-based policy (as opposed to arbitrary targets-based policy), and basic science and numeracy.

[Dipper]

That is an unbelievably silly comment.

Bob B goes head to head with J K Galbraith! Bring it on!

[Bob B]

"Bob B goes head to head with J K Galbraith! Bring it on!"

Yet another unbelievably silly comment because:

- it avoids the basic issue of how governments or central banks with inflation target remits are inescapably committed to forecasting. Of course, there can be forecasting errors and policy mistakes but the fundamental issue is always whether there is some better policy framework, which is effective in securing objectives but which doesn't depend on elements of forecasting.

- Pres GWB kept on and on about Free Market Capitalism and is now pressing an interventionist policy to rescue GM and Chrysler - how come? Why not just let the two companies go to the wall so Free Market Capitalism can prevail?

- it argues by authority and not about the issues. I never had much regard for JK Galbraith anyway - even when he was still alive it always puzzled me as to why the BBC Today Prog kept selecting him to represent the economist viewpoint. There were and are many better economists.

[Bob B]

Btw JK Galbraith was a self-proclaimed "keynesian" and I would dearly like to know how any government committed to keynesianism could perform fine-tuning of aggregate demand by fiscal policy interventions so as to maintain stable employment without engaging in forecasting.

[Frank H Little]

GWB has been pressed into supporting GM & Chrysler against his natural instincts, as his speech announcing the deal made clear.

[Bob B]

"GWB has been pressed into supporting GM & Chrysler against his natural instincts"

That seems a bit perverse - what will the Religious Right say about going against "natural instincts"?

If letting GM and Chrysler go to the wall was so obviously the better course, why not let that happen to demonstrate the public benefits of Free Market Capitalism?

In all, it makes GWB look even more of a dumbo to external observers.

[Richard Lawson]

You must admit it is enjoyable when experts get it wrong. There's definitely more schadenfreude about these days than there used to be.

[Tom James]

@Frank H Little: "GWB has been pressed into supporting GM & Chrysler against his natural instincts, as his speech announcing the deal made clear."

Well it depends on what his natural instincts actually are - if they are to preserve individual liberty and competitive markets then he certainly is going against his natural instincts.

However if his natural instinct is to "help out my rich buddies" then he is doing exactly what he is comfortable with.

[Bob B]

"You must admit it is enjoyable when experts get it wrong"

Never mind economists, when did politicians, lawyers, medics, scientists, historians, philosophers etc etc stop disagreeing?

Btw The [London] Times convened a panel of experts to assess 42 past US Presidents.

Going from top to bottom, GWB was rated in the bottom ten at number 37, equal with Richard Nixon:
http://www.timesonline.co.uk/tol/news/world/us_and_americas/us_elections/article5029204.ece

That's rather unfair IMO. Whatever else, Tricky Dicky was regarded as smart, if anything too smart at evading the straight and honest course, whereas GWB is routinely portrayed as a dumbo in political cartoons and comedy shows. The response of studio audiences suggests they find that portrayal of Bush really cool.

It can be sad when experts are correct. Some researchers have repeated Stanley Milgram's (infamous) staged experiment in the 1960s where he demonstrated that ordinary people will commit acts of violence that conflict with their personal conscience and moral convictions if instructed to do so by an authority figure. The bad news is that after repeating the experiment, the researchers came up with similar results:
http://abcnews.go.com/Health/MindMoodNews/story?id=6496911&page=1

We should beware of authority figures and arguing by authority for fear of where that may lead.

[Guy Herbert]

And since someone raised Milgram, one should note that torture is popular as providing confirmation - eventually - of whatever the torturer's boss firmly believes. Another form of expertise vindicated!

[Dipper]

Obviously a sore point Bob B.

Surely the issue is not whether governments need to forecast economic growth, but would they be be better off using astrologers rather than economists?

For what its worth (not a lot) I would have thought the main use of economists is not an average point forecast, but to warn which side of the forecast is the one you should worry about.

[Dipper]

and, whilst on the subject of Galbraith, whilst I cannot judge his performance as an economist, his analysis of the world of business and how it operates is by far the closest I've come across to the world I live in.

[Bob B]

"For what its worth (not a lot) I would have thought the main use of economists is not an average point forecast, but to warn which side of the forecast is the one you should worry about."

Try, for example, the Bank of England's Quarterly Inflation Report for May 2008 - the latest report for which I could easily find a link:
http://news.bbc.co.uk/1/shared/bsp/hi/pdfs/14_05_08inflation.pdf

The "fans" in graphs therein of projected time series in the Charts 5.x are intended to indicate confidence levels.

HM Treasury regularly publishes surveys of independent forecasts of the UK economy so readers can follow their favourite source or judge which has the better track record:
http://www.hm-treasury.gov.uk/d/200812forcomp.pdf

Usually, the NIESR publishes in its Economic Review periodic reports on which forecaster (incl HMT) has been producing the best forecasts and - importantly - analysing why. The FT usually has an annual news report on the "best" forecaster - although some tend to be good at one time series but not at another.

With that fairly transparent background, it's unreasonable to complain that observers of the UK economy are being presented on a take-it-or-leave basis with a single forecast shrouded in mystery. One current problem is that forecasting models inevitably have to be calibrated on past trends and the causes of the present crisis - pervasive failures in leading financial institutions and the financial systems of the US and the UK - are unprecedented in the post-war period. That means current forecasts are (unavoidably) subject to an unusual degree of uncertainty.

Of course, forecasts are subject to error but forecasts are inescapable for governments, local authorities and for central banks with inflation target remits.

Rubbishing all economic forecasts leaves unanswered the basic question as to what policy framework, untainted by forecasts, would yield better outcomes? After all, Free Market Capitalism with minimal regulation is why we are where we are.

[Alex Martelli]

I'm not an expert in economics (rank if enthusiastic amateur there -- my actual professional field is software development), but the key insight in this post, applied to what customers demand from experts in MY field, was an eye-opener -- I commented on that in detail at http://aleaxit.blogspot.com/2008/12/software-development-is-not-yellow.html

[Dipper]

so if economic forecasts are essential to the efficient running of government, how did the Romans manage to run an empire without them? Or did they have a similar function somewhere? (Not being sarcastic - a genuine question)

[Bob B]

"so if economic forecasts are essential to the efficient running of government, how did the Romans manage to run an empire without them?"

A short answer: the Roman Army, bread and cicuses in Rome for the plebs to keep them contented - the bread depended on wheat tributes from the empire enforced by the army - and slavery.

Admittedly there are latter day echoes - the current focus on football and the 2012 Olympics in London instead of circuses - but slavery and tributes don't figure.

The relevant current comparison is not with the Roman empire or even with the (mostly admirable) administration of Tudor England but with active policy frames in other affluent countries nowadays - the OECD countries.

The challenging historical issue is how come industrialisation was pioneered in Britain without state direction and with minimum state control over the realm? Micro-management from Whitehall wasn't an option: incoming data about the pace and nature of economic activity was mostly from the tardy arrival of stats from customs and excise and inland revenue and from reports by itinerant journalists.

Historians are still debating how and why Britain pioneered industrialisation - new books are still being published about the necessary conditions for starting industrialisation. If you read into this, you'll find that other west European nations did not follow the British model but resorted from the start to more interventionist roles for government [eg Clive Trebilcock: The Industrialisation of the Continental Powers 1780-1914 (Longman 1981)].

Also note this important insight:

"We have noted a substantial body of original research . . . which found that stagnant or declining literacy underlay the 'revolution' of the late eighteenth and early nineteenth centuries. . . Britain in 1850 was the wealthiest country in the world but only in the second rank as regards literacy levels. [Nick] Crafts has shown that in 1870 when Britain was world economic leader, its school enrolment ratio was only 0.168 compared with the European norm of 0.514 and 'Britain persistently had a relatively low rate of accumulation of human capital'."
Sanderson: Education, economic change and society in 1780-1870 (Cambridge UP, 1995) p.61

We still have human capital problems now !

[Jon]

@ Ian

Granted that McIlkenny v Chief Constable of West Midlands Police Force was not Lord Denning's finest hour. However:

1) He never said that "It is better to keep innocent men in prison, than to let them go free and bring the system into disrepute."

2) The comments have to be taken in context -- at that stage, the men, having exhausted the criminal appeals process, were seeking to use the civil courts to retry an issue that had been heard in full at their first criminal trial. It would have set a very dangerous precedent had the case been allowed to go ahead.