The trouble with experts

Paul raises an important issue: what’s wrong with experts? I suspect the answer is: plenty.

These many faults, however, don’t include the failure to foresee the financial crisis. It’s not the job of economists to act as soothsayers, any more than we expect mechanics to predict the next fault with our car, or doctors our next illness. There’s a lot of good economists can do without prophecy.

Instead, one problem is that experts – in public at least – don’t sufficiently identify what Charlie Munger called the edge of their competence: they don’t say what they don’t know. The problem with economists isn’t so much that they didn’t foresee the crisis as that they failed to tell the public that recessions are unforecastable (at least by mainstream techniques?) In 2000, Prakash Loungani reported that the economic consensus had failed to predict almost every recession, and AFAIK, not much has changed since.

Of course, there’s nothing unusual about economists in this regard. Gerd Gigerenzer said that doctors are “often wrong but never in doubt”; bosses rarely openly discuss the limits on their ability to control business; bankers don’t often admit to being unable to manage risk; fund managers don’t publicly say that markets are largely efficient; and so on.

I fear that academics’ pressure to publish exacerbates this error. As Andrew Gelman has consistently argued, the “p<0.05=result!” mentality has given us a lot of bad science. This can weaken public confidence. One recent example of this is the claim that roast potatoes cause cancer. This ran into the common-sense problem that everybody eats roasties but only a few get cancer, which tells us that the link, if any, must be weak. It’s this sort of thing that generates the sentiment “Experts – what do they know?”

In this context, we should remember a point made by William Easterly – that “experts are as prone to cognitive biases as the rest of us”. One of these is overconfidence. Another is that groupthink can generate intellectual fashions, such as perhaps the dominance of DSGE models.

Perhaps another facet of this overconfidence is the presumption that knowledge and expertise must be centralized, within a single mind. But as Hayek pointed out, in some contexts this is not the case. Instead, knowledge can be fragmentary, tacit and dispersed. Consumers, for example, might be better economic forecasters than the experts – as might bond markets. And workers might sometimes know more than bosses.

Which brings us to a big problem. We are too often given the impression that experts are on the side of power. In part, this is the fault of the media: bosses, for example, are routinely presented as a priestly elite rather than as mere rent-seekers.

But the fault also lies with experts themselves. Economists fail to stress sufficiently that some of the settled findings of their discipline undermine claims to power. For example, the efficient markets hypothesis tells us that fund managers are charlatans; the prevalence of diseconomies of scale suggests that bosses aren’t as much in control as they pretend; and public choice economics warns us that those in power are often just rent-seekers.

In this context, even an otherwise admirable empiricism can be reactionary. Studying the effects of the small actual and feasible tweaks to capitalism means the question of big changes gets marginalized. In this way, as Paul says, economists unwittingly accept capitalist realism (pdf).

Related, to this, mainstream economists have been insufficiently awake to distributional issues. The default ethical criterion in economics is Pareto efficiency. So, for example, we’ve accepted globalization because it is potentially Pareto efficient: the gains from it to western economies are bigger than the losses. This has blinded us to the fact that a potential Pareto improvement in which the losers don’t in fact get compensation is a much more dubious matter.

If economists had a different default mental model, they’d have been better able to press the case for policies that really do benefit everybody. If economists used Rawls’ difference principle – which stresses the need for policies and institutions that “are to be to the greatest benefit of the least advantaged members of society” – they’d be much more conscious of inequality.

We all know that a gap has emerged between experts and the public – expressed in that heckle, “That’s your bloody GDP. Not ours.” If the quality of public discourse is to be lifted out of the gutter, this gap must close. At least part of the onus on doing so lies with the experts themselves.

Comments

[Patrick Kirk]

I'm struggling to put into words here. A lot of problems with "experts" is that the expertise is levered into wrong setting. Economists are asked to pontificate about Brexit which is a primarily social decision. Businesssmen get elected to run countries which they have zero relevant experince to qualify them to do. Its like some kind of halo effect which says "This woman is an expert about subject A so lets assume the expertise alos applies to subject B." And when the "expert in subject B" fails, their expertise in subject A is also discreditted and all other experts discreddited aloing with them.

I have no idea if this is fixable.

[Luis Enrique]

"These many faults, however, don’t include the failure to foresee the financial crisis."

I would really like to persuade you to change your mind about this!

Economists can warn about the dangers of large and persistent deficits (so they can warn about public finance crises) and ditto for current account deficits. Why don't you think they could and should have done the same for the banks?

Economists have since published papers looking at what seems to precede crises*, they could have been aware of the dangers in banking and monitored development closely and warned of danger ahead.

The problem with what I am describing is that we would spend years with the financial sector warning system flashing red, predicting a crisis that does not arrive (until it does) and who knows, maybe even pointing to risks might have changed behavior to prevent crises so we would now be writing about how economists are terrible because they predict crises that don't arise. But that possibility aside, I still don't understand why "you guys should have been on top of this" is not a legitimate criticism of economists.

* here are some high status authors doing that. Whilstles and bells aside, how these differ in essence from what Steven Keen produced, I don't know.
http://economics.ucdavis.edu/people/amtaylor/files/w21486.pdf
http://economics.ucdavis.edu/people/amtaylor/files/w20501.pdf
http://economics.ucdavis.edu/people/amtaylor/files/w21039.pdf

[Matthew Moore]

The Guardian did a really good piece on this in their 'Long Read' section recently. The thrust of it was that experts in all fields are similar in a set of core dimensions: education, employment and income, which means that they sometimes seem like a vested interest protecting their privileges. The experts were mostly in favour of Remain, but we know that they mostly belong to a class for whom EU membership is individually beneficial.

'Experts' are now seen similarly to the CBI and the Unions: cherry-picking blowhards on the make.

PS did you deliberately get the P value quote wrong to make your point?

[Luis Enrique]

If economists used Rawls’ difference principle ... they’d be much more conscious of inequality.

well ditto surely for doing simple utilitarian welfare policy analysis and the assumption of diminishing utility of consumption. We're always told this is a no no but imo it's the place to start. "Assume everybody is the same but that a £ means more to a poor person than a rich person" seems like an excellent baseline to me.

[Luis Enrique]

(though on the Pareto compensating losers thing, I think that Vollrath is spot on to say that monetary compensation is only half the story: https://growthecon.com/blog/Trade/
real expertise in "compensating losers" requires behavioral aspects. )

[Blissex]

Nearly all experts have careers paid for by an employer or need funding to do their research. Those who speak out publicly usually are explicitly or tacitly advocates for some lobby, or ideologically biased.

The public knows very well that two different experts with different incentives or biases can come up with radically different explanations and policies.

[Jim]

"The Guardian did a really good piece on this in their 'Long Read' section recently. The thrust of it was that experts in all fields are similar in a set of core dimensions: education, employment and income, which means that they sometimes seem like a vested interest protecting their privileges. The experts were mostly in favour of Remain, but we know that they mostly belong to a class for whom EU membership is individually beneficial"

Precisely. Experts in every fields have risen to the top on the back of the status quo, and have a vested interest in the status quo continuing. They have careers, prestige, salaries, pensions, often provided by the very State they are supposed to be providing with independent advice, all tied up in their pronouncements, it is not humanly possible for such vested interests to not cloud their judgement. For example its noticeable that many of the scientists who oppose the whole 'Climate Change' issue are retired ones. Freed of the shackles of career and wage packets to maintain, they feel free to express views they undoubtedly held while in office, but were afraid to make public.

[Luis Enrique]

Jim, Blissex I think you're wrong. For all their faults, academic economists of my acquaintance just say it as they see it. If you think career incentives matter it's subliminal, but you can turn that on anyone, including iconoclasts who get high smelling their own farts. No doubt people like Portes or Wren-Lewis accumulate positions over time they may find it hard to deviate from, but that's people for you, not experts per se.

[chris]

@ Matthew - that was a typo, now corrected.
@ Luis - I think you highlight the problem when you say "we would spend years with the financial sector warning system flashing red". We all know (or should) that crises happen: the difficulty is knowing when.
Some simple stock market lead indicators (sell in May, foreign buying of US stocks, and the ten-month average rule) did warn us of falling shares in 2008. So they got the timing right, but without any insight into the nature of the crisis.
I think this just vindicates Elster's point, that explanation and prediction are very different things.

[Luis Enrique]

Chris,

But warning of a crisis does not mean getting the year right. Economists would be in much better standing right now if they had spent the decade before 2007 saying "look at what's happening to leverage! danger!" - Economists could have paid more attention to the state of bank balance sheets. Why isn't "not paying attention where it was needed" a failing?

Those Taylor papers I linked to use perfectly mainstream techniques and if economists had been paying attention it would have been as easy for them to fret publicly about financial risk as they do at the drop of a hat about government deficits etc.

[Blissex]

«but that's people for you, not experts per se.»

Indeed I agree; my point is that experts too are subject to manifest or "subliminal" incentives, and they too are ideologically biased, indeed like ordinary people, and therefore, deliberately or inadvertently engage in advocacy. I am not even making it an issue of bad faith: the Economist at the CBI who supports employer-benefiting with his expertise may well be entirely sincere and persuaded of the validity of her opinions.

After all in court when there is a technical question every side is entitled to their own expect, and sometimes the court (in some countries) appoint their own expert too, to hear contrasting opinions.

That is at the *individual* level. Then there is the aggregate level And here the situation is quite different, because there are collective, institutional biases. Some political sides somehow have a lot more experts than others, and strangely enough those experts as a rule support the policies advocated by their side.
Perhaps it is because the policies advocated by that side are indeed "the best", but if 50 CBI-side Economists tell me something and 2 employee-side political economists tell me differently, as an employee maybe I don't jump to the conclusion "oh wow the CBI must be right" :-).

[Jim]

Then there's also the issue of testing the expertise. An expert surgeon is pretty easy to test, do more of his patients die than comparable surgeons? Do the aircraft an aircraft designer designs crash and burn when they get off the ground? Does an architects buildings reduce to a pile of rubble in the first storm? All eminently testable, and experts in those type of fields would be drummed out of their professions if their output was consistently wrong, and lose their right to practice.

However the type of experts we are talking about can be wrong repeatedly, and no-one drums them out of anything. Do people still allow David Blanchflower to prognosticate on economics, despite his utter failure in prediction, having predicted 5m unemployed if the Tories proceeded with their budgetary plans in 2010? Why yes of course they do, he's a Professor of Economics in the US.

This is why no one believes 'experts' in the public policy field - they get it wrong repeatedly and continue to get paid big bucks nonetheless. A lorry driver who made that many errors would probably be in jail.

[Keith]

Jim is half right, but only half. For example patients die all the time, it does not follow the doctor or surgeon are to blame. Some errors in surgery can be easily detected, but the death rate cannot tell you anything unless you take into account the condition of each patient before surgery. Some patients are quite fit others very ill or impaired with "comorbidities" so much more likely to die. Any building will collapse if it encounters a strong enough force; a designer makes assumptions about what forces a structure can be expected to encounter when designing the building or machine. They are not to blame for failures if the conditions leading to failure could not be reasonably assumed in advance.

I think Luis Enrique has a fair point. My view is that the failure to anticipate the GFC is an example of ideological bias. Most economists took a relaxed view about bank leverage as the banks are private sector bodies assumed to be efficient as disciplined by "the market". History is full of examples of crisis arising in the private sector. Until post world war 2 economic depression always arose from private sector failure. Without trying to predict exactly what is going to happen economists should lay down guidelines for private sector actors to reduce the danger of instability. And it should be a goal of experts and governments to take precautions against systematic failures. Which is what Keynes for example advocated for decades.

[CecilTheLion]

As Mr. Galbraith put it, "The only function of economic forecasting is to make astrology look respectable."

Actually I wanted to use this post to ask you for a Marxist take on Scott Alexander's recent article on cost disease, which is making quite the buzz. I'll link it here: http://slatestarcodex.com/2017/02/09/considerations-on-cost-disease/

[B.L. Zebub]

@Chris

These many faults, however, don’t include the failure to foresee the financial crisis. It’s not the job of economists to act as soothsayers, any more than we expect mechanics to predict the next fault with our car, or doctors our next illness. There’s a lot of good economists can do without prophecy.

Nonsense, Chris. You are employing word games, semantics, in an attempt to defend the indefencible.

When H.M. asked why didn't anybody see it coming, she didn't use the word "forecast". I know that, so do you. Everybody knows that.

She was giving voice to millions of people the world over: why didn't anybody PREDICT a crisis?

She didn't ask for a precise forecast (as in "Lehman Brothers will go declare bankruptcy on September 15, 2008 at precisely 9 AM and its CEO will be using pink underwear, after taking 8.701 cups of coffee, black no sugar") nor one was needed. What was needed was a warning.

--------

A friend of mine died of cancer last year. She had a brain surgery in January. The neurosurgeon has happy with the outcome: she wasn't cured, but she would have a few more months of relative health. He ** predicted ** she should have at least 6 months, but it was unlikely she would last a full year. Towards the end, depending on chemo and radiotherapy, she could experience several problems: from headaches, to mood changes; from loss of mobility to loss of control over her bodily functions.

Nobody demanded a forecast: as in on which date she will die?

She died four months ago. She did experience many (not all) of the conditions the surgeon said said she would. His prediction was right.

--------

What I find irritating in that kind of defence of economics is that I am sure you both mean well and know better. And yet, you insist on that. You are better than that.

[DP from Durbs]

Luis Enrique,

Even private debt twitchers like Steve Keen have accumulated a long list of unfulfilled prophecies since 2008.

And then there's the doom mongers that have reliably predicted 10 of the last 2 meltdowns.

[Dipper]

my understanding of the GFC is as follows:

- there was a massive credit boom through the housing market. This was basically a pyramid scheme which as with all successful pyramid schemes stopped when there was no-one left to bring in to the scheme. This was encouraged by politicians who benefited from the growth in GDP fuelled by borrowing. Economists pointed this out quite a lot at the time.

- all banks are bust all of the time, in that if all the creditors appeared at once and asked for their money the banks would not be able to find it.If they could find it, they would be investment funds not banks.

- creditors do not generally do this as it requires co-ordiunation amongst them to go after a single bank. This started to happen in 2007/8 when the investor community started to line up banks in order of perceived vulnerability and went after them one at a time.

- the rout of banks was halted when politicians under the leadership of Gordon Brown publicly stood behind banks. If politicians had done this at the outset there would not have been a financial crisis of banking insolvency. There would have been other significant repercussions, but the point is that politicians had the means to stop the GFC in its crash and chose not to do so.

I'm fairly sure economists pointed out much of this before hand, during, and after, but the path taken was down to decisions made or not made by politicians. In this sense, comparisons with doctors who inform you of consequences of behaviour and likely progressions but are not ultimately responsible if you choose to take hard drugs/starve yourself/binge eat are entirely accurate.

So I think economists come out of the GFC quite well. And I say this as someone who regards economists as people who lacked the brains or courage to tackle the One True Subject - Physics.

[Luis Enrique]

DP

yes. I also get irritated by everybody who predicted some sort of debt-related crisis claiming credit for predicting the GFC. If you ask me, the most important thing about the GFC was not merely that banks suddenly realized they had more bad debts on their books than they thought, but that they managed to get themselves into a state (shadow banks, derivatives and all that) that what might have been a relatively minor problem (a higher than anticipated rate of mortgage defaults) was transformed into an system-wide extinction event. Well, I exaggerate.

To an extent you can claim credit by claiming the banks were going to get themselves into trouble because some broad risk indicators were getting elevated, without having predicted quite how - and how severely - it would happen. But I think if what you really had in mind was a 'normal' bad debt related financial crisis, you really ought to admit how you didn't see much of what actually happened coming.

[Tynnie Todgers]

"These many faults, however, don’t include the failure to foresee the financial crisis. It’s not the job of economists to act as soothsayers, any more than we expect mechanics to predict the next fault with our car, or doctors our next illness."

When a mechanic says your brake pads are worn, or a doctor that your blood pressure is high, predicting the next fault or illness is exactly what they're doing - and exactly what we expect. It isn't just that economic orthodoxy - "experts" - didn't see the GFC coming, but that they can't even offer a decent diagnosis after the fact.

[Luis Enrique]

"they can't even offer a decent diagnosis after the fact."

that's just plain false. you've got Gary Gorton.

https://global.oup.com/academic/product/misunderstanding-financial-crises-9780199922901?cc=us&lang=en&

Sufi

http://faculty.chicagobooth.edu/amir.sufi/houseofdebt.html

Admati Hellwig
http://bankersnewclothes.com/

Shin
https://global.oup.com/academic/product/risk-and-liquidity-9780199546367?cc=us&lang=en&

his Northern Rock paper is excellent
https://www.aeaweb.org/articles?id=10.1257/jep.23.1.101

Brunnermeir
https://www.aeaweb.org/articles?id=10.1257/jep.23.1.77

these are just off the top of my head.

the GFC was complicated, had many causes, any one account likely to be partial, but one thing economists definitely cannot be accused of not being able to explain what happened.

[Jim]

"The GFC was complicated, had many causes, any one account likely to be partial, but one thing economists definitely cannot be accused of not being able to explain what happened."

Very true, but being able to pick through the entrails to tell you exactly what killed the patient isn't much use to the living. If the best economists can do is tell us what happened after the event, they should STFU about the future and concetrate on the past. Not so much money and prestige in that though................

[Luis Enrique]

Jim we know they failed to predict, but these same economists now providing after the fact diagnosis are also providing ways of avoiding a repeat occurrence. Only some of which is being adopted because the bank lobby is too strong and politicians too uninterested.

[Dipper]

@Luis - how is the bank lobby too strong? Last time I checked they were implementing lots of regulations - at the cost of doing any other work - and complying with all requirements. Can you give some examples?.

[William Meyer]

It's odd that you, as a Marxist, aren't more sensitive to the idea that the expertocracy might well have become a "class", and have "class consciousness", and "class interests." After all, social relations are a result of the means of production, and experts are definitely a major part of the means of production, right?

So, to allow me my hypothesis, why might people who are not in this class distrust the experts, and even think that the main agenda of the expertocracy might be ideologies that support and sustain the power and privileges of the expertocracy? Which, by the way, includes not only the professions, finance,media, political lobbyists and also senior corporate managers, all of whom today have been completely "professionalized" (via the same schools and general life style.)

You have actually heard of "the new middle class" (a.k.a. the technocracy and the expertocracy) haven't you? It's an idea that's been around for nearly 100 years in history and sociology.

[joe]

The discussion is getting into confounding three threads, expertise, economics, and cognitive biases.

If you take a weather forecast of 30% chance of rain tomorrow. The forecaster has used her expertise and used an ensemble of models that look at the issue from many perspectives. She uses her expertise to take notice of small clues that things might not be as per usual. She has no bias but uses her skills honestly and as best she can.

The forecast has provided a percentage chance based on past evidence in similar circumstances and a fixed date for the forecast to apply on.

It does not rain and the ill-informed naysayers say they got it badly wrong - they were expecting some rain. The informed say they got it right as they forecast it would be fine with a 70% probability and that was the most likely outcome..

The non-expert uses her seaweed or hall barometer. She may be right, and if she is, thinks she knows more than the experts at the Met Office.

There were many who were getting wary before 2008. Risk managers told their senior management in no uncertain terms but were brushed off or fired. Politicians despite advice did not want to kill off the feel good factor. Some banks made sure they passed on the toxic mortgage bundles as quickly as they could - knowing that someone else would eventually pick up the bill. The public were getting concerned as every night they would get home and some bank would be offering pre-approved loans of £50,000, no questions - too good to last was the common sense. No one knew when the final events would happen or their extent because there was no transparency. In any case, recessions and crashes occur at some point on a regular basis. So anybody can forecast that if they leave the date out.

Economists have to face up to it that they have no expertise in forecasting. Every month their consensus business forecasts are way out even on simplistic and simple matters. Month after month they never take steps to improve. You can make a good living simply going against their forecasts. Pundits are the second worst forecasters as they do have the full range of cognitive biases to defend a position taken, and always to try to look clever by not predicting the obvious. Pushing the regression line forwards is not forecasting.

Forecasting is a specialist skill requiring prediction expertise, a wide ranging enquiring mind that looks at many issues other then financial ones and from many perspectives, fighting bias at every stage. They do discuss with other forecasters as they will invariably have missed some key aspect. This seems to be the total opposite of what lone economists do with their simplistic, pet theories and why they are invariably wrong.

[Luis Enrique]

"Economists have to face up to it that they have no expertise in forecasting .. Month after month they never take steps to improve."

really? what's this: http://voxeu.org/article/improved-approach-empirical-modelling-0

[andrew adams]

For example its noticeable that many of the scientists who oppose the whole 'Climate Change' issue are retired ones. Freed of the shackles of career and wage packets to maintain, they feel free to express views they undoubtedly held while in office, but were afraid to make public.

The basic problem with this argument is that their views are nonsense. Nonsense views do tend to get a rough ride in academic circles, it's nothing to do with the fact that they challenge established opinion.

[derrida derider]

I'm surprised that none of your commenters haven't pointed out that you have confused
Pareto efficiency with the Kaldor-Hicks criterion.

A Pareto improvement is where someone is better off with no one actually being worse off. A Kaldor-Hicks improvement is as you describe - someone is better off even after they fully compensate those made worse off. In real life Pareto improvements are vanishingly rare while Kaldor-Hicks improvements are very common but empirically uninteresting unless you have some mechanism by which the compensation happens.

This is in fact the nub of debates over such things as globalisation and Brexit because international trade gives (often large) Kaldor-Hicks, not Pareto, improvements.

[derrida derider]

joe, the bulk of economists do not do any forecasting at all, at least about the macro economy. One of the main things economists get taught in grad school days is that in fact the macro economy is pretty much innately unforecastable ex ante, whether by formal models or "expertise". The standard in-joke is Samuelson's "economists have predicted seven of the last three recessions".

The ones you see prognosticating on the TV are mostly those paid by their employer (ie banks and similar) to talk their employers' book. See Chris' comments about fund managers, of whom they are often just adjuncts.

[Dipper]

1. Another chance to recommend "All The Devils Are Here"by Bethany Maclean and Joe Nocera. A comprehensive account of how the whole house of cards was constructed and then collapsed.

[Dipper]

2. And Climate science is quite an interesting case of experts and their egos. the temperature record is at the moment on the margin of having significance. Climate scientists were caught out by the hiatus, and they currently have two explanations for it - it was due to deep ocean currents taking in heat which is released in major El Nino events like the one we just had, and the other explanation is that there wasn't a hiatus.

The thing that sticks out like a sore thumb is the level of CO2 in the atmosphere which has broken out of a half a million year stretch of between 180 ppm and 270 ppm to be now 400 ppm. The volume of the increase being about half what man has released since the start of the industrial revolution. And everything we know about physics tells us that means the planet will warm up.

The discussions about temperature are at best discussions of the rate of change, and at worst just arguments about who has the right to have their voice boom across the world of climate politics. I'm constantly amused/disturbed by the number of people who weigh in on arguments about temperature when CO2 levels are the big warning sign.

[Tynnie Todgers]

@Luis Enrique - AFAICT from the book reviews & papers you've linked to, the authors challenge the expert consensus which existed before the fact and have not reached one since. That was my point. I didn't say no economist has attempted a diagnosis after the fact - inevitably they're falling over each other to do so.

[Luis Enrique]

come on, those accounts may differ in emphasis but afaik they are consistent and they add up to "a decent diagnosis"

[Tynnie Todgers]

It's rather beside the point, but I don't think they do. I see two of several competing narratives among that lot. One broadly has it that a financial wobble blew up into a crisis due to the interconnectedness of highly levereged banks. The other posits a systemic problem of household debt/lack of qualified borrowers whereby bailing out debtors could have averted crisis.

Still others out there range from gov't-did-it (excessive regulation, CRA etc) to capitalism-did-it (inherent contradictions a la Marx etc).

The point remains that they're at odds with the previous expert consensus that rational actors and market evaluation of assets had precipitated a new era of stability - and that there's no sign of any new paradigm.