Why idiots succeed

Several tweeters, such as Anna and Hopi, have expressed dismay at the fact that Iain Duncan Smith has kept his job in the cabinet (re)shuffle, despite his gross inadequacies. His survival, however, highlights an important fact - that, sometimes, organizations and markets can actually favour incompetence.  There are (at least) eight mechanisms through which this can happen.

1. The wet bed. If a man has pissed the bed, you don't ask someone else to sleep in it. Cameron might well have thought "Universal Credit is an immense clusterfuck which nobody can sort out in a few months. Why waste the reputation and morale of a good man by asking him to try?" I suspect similar thinking led Luiz Filipe Scolari to keep David Luiz on the pitch during Brazil's 7-1 mullering by Germany.

2. A disposition effect. Every bad employee is a bad hire. However, a combination of ego involvement ("this was my decision so it must be a good one") and the endowment effect ("we've got him so he must be good") stops hirers from immediately realizing their error. Just as stock market investors tend to hold onto bad stocks, because of their refusal to admit error, so employers hang onto bad staff.

3. Noise vs signal. In many contexts, feedback about performance is noisy. Ran Spiegler shows (pdf), this can create a market for quacks. For example, if you would have recovered from some ailment anyway but take a homeopathic treatment before the recovery, you can easily convince yourself that the treatment worked. And you'll tell others. A similar mechanism can favour the fund manager who rides a bull market, or the corporate boss who is lucky enough to get a job at a firm with good organizational capital.  

4. The devil you know. In many jobs, a worker's ability can only be assessed after he has done it. As Marko Tervio has shown, this alone can generate adverse selection; the mediocrity who has a track record that is just adequate will be preferred to the unproven man of potentially greater ability. In itself, this is a story about markets favouring mediocrities rather than incompetents, but mechanisms 2 and 3 above suggest that the bar for mediocrity might be set so low as to allow idiots to thrive.

5. Survival of the unfittest. Bjorn-Christopher Witte describes how, sometimes, competition between fund managers can encourage reckless risk-taking with the result that lucky chancers rather than the genuinely skilled will thrive. For example, in the early 00s bankers who danced to the music and took risks got big bonuses whilst those who sat it out got sacked. And during the tech bubble money flowed to those managers who thought boo.com and Baltimire Technologies were good stocks, whilst sceptical fund managers such as Tony Dye were fired.

6. Desperation. If people are desperate for a very high pay-off, they'll be attracted to incompetents and fraudsters, as only these are stupid or criminal enough to offer such rewards. As Laurie has said, "sometimes when you’re dying of thirst, you have to drink the Kool-Aid." This is why con-artists often prey upon the terminally ill or bereaved. But it also lies behind what I've called the Bonnie Tyler syndrome - the urge (often on the left) for a great hero.

7. Product differentiation. In a wonderful paper (pdf) on the persistence of the market for quack medicines in the 19th century, Werner Troesken points out that the manufacturers of such remedies spent fortunes on advertising and product differentiation. In this way, the failure of one medicine did not discredit the industry, but merely shifted demand to other quacks. I suspect the fund management industry operates on a similar principle. So might politics: insiders want us to think that Paterson, Fox, Hammond or whoever are distinguishable when in fact they are fungible.  

8. Like hires like. Senior managers like to hire people like themselves not just because they want yes-men, but because (say) financial people find it easier to assess finance skills and engineers find it easier to assess other engineers. Knowing this, underlings hoping for promotion will cultivate the skills their bosses have, even if their comparative advantage lies elsewhere. The upshot of this, according to Eric Hughson and colleagues, is that organizations can eventually be run by second-rate MBAs whilst technical skills are completely weeded out. Joao Ricardo Faria shows (pdf) that this sort of mechanism can generate the Dilbert principle.

Now, I stress that these mechanisms are not universal, although they are sufficiently widespread to explain the obvious fact that some idiots and charlatans survive and thrive in many organizations. It is, of course, the case that competition does sometimes improve quality. But this Econ 101 view is only a partial truth. The notion that we live in a meritocracy - or even that a meritocracy is possible - is just...

Comments

[Dave Timoney]

Given the ample evidence of Cameron and Osborne's prior contempt for IDS's intellectual and managerial abilities, I don't think this can be explained by any theory that rests upon their delusion.

More likely is that IDS remains in post because he is, in their eyes, a success. Perhaps we've just misunderstood that the brief was to knacker the benefits system so that claimants increasingly give up, while shovelling large amounts of public money to favoured suppliers.

He's certainly done a good job in proving the incompetence of the state in managing services.

[Phil Beesley]

"Given the ample evidence of Cameron and Osborne's prior contempt for IDS's intellectual and managerial abilities..."

I follow the argument. If IDS was a government duck, he resided there for donkey's years. Why? If he was incompetent, why did he continue in the job?

[Thornton Hall]

Number 6 describes the econoblogosphere to a t.

[Thornton Hall]

I meant number 7!

[Blair]

Brilliant and accurate! Especially 8.

[Roob]

It's surely the brown envelope of incrimination information in IDS's safe?

[Keith]

It's surely the brown envelope of incrimination information in IDS's safe?

Or the "dirt book"

[Cwhope]

Surprised this doesn't mention the Peter Principle: incompetents stay in their jobs because if they were competent they would have been promoted.
@cwhope

[Stevenclarkesblog.wordpress.com]

How easy is it to get reliable feedback from successful Minister-ing anyway?

In the private sector, there are easily measurable financial results, but I'm not sure if there are any in the Government.

The voters get to give feedback - but all Gov't services are bundled together so there's little chance of judging specific policies or Departments.

A lot of the feedback comes from the press, which is all tribalism, prejudice etc and very rarely can judge on the technical success of this policy or that outside of the quality press and certain blogs.

[Jeffrey678]

Nepotism.

[Abdullah]

He's a right-y. That seems to be complexion of the cabinet now.

[Jonathan]

The first-rate hire the first-rate but the second-rate hire the third-rate.

[An Alien Visitor]

"incompetents stay in their jobs because if they were competent they would have been promoted."

There is another argument that says only the incompetents get promoted precisely because they are no good at their job!

[Jeff]

Incompetent at what? Often what we want is not results, but rather something like feel-goodness. Some are good at providing that, irrespective of their job description.