On selection mechanisms

Market research is vital. Had the breakaway six done some, they’d have known it was an unpopular idea and so wouldn’t have destroyed their bargaining power with UEFA and their (in some cases tiny anyway) popularity with fans.  

Market research is a menace. Sir Keir Starmer’s concern with focus groups is creating timidity and a lack of direction. It’s causing him to reject Wayne Gretzky’s advice to skate to where the puck is going to be rather than to where it has been. Henry Ford never actually said “if I had asked people what they wanted, they would have said faster horses.” But it captures a limitation to how much we should listen to market research.

Which of these paragraphs is correct?

Both. What we have here is an example of Niels Bohr’s saying: the opposite of a great truth is another great truth.

We should think of all decision-making systems – be they corporate management, research methods, government regulation, our own mental habits and so on - as selection mechanisms.

Sometimes, such as in politics, these selection mechanisms select the worst, leading to inadequate politicians and bad ideas. Other mechanisms, though, can select against the worst whilst not necessarily selecting the best. I suspect this is true of market research. It’s perfectly possible that market research would have told the breakaway six they were backing a turkey whilst at the same time not telling Labour what will win most votes: one reason for this could be that people don’t actually know their preferences, and certainly not their future ones.

If this is the case (and this post is not really about that) then what we have here is one of a set of decision-making systems which function better at weeding out egregiously bad results than they do as selecting for the best ones.

Perhaps the best-known example of this is the US constitution. Its function is more to protect against tyranny than it is to ensure the best possible government, which might on occasion require a more activist state than the constitution permits.

Another example is Friedman’s idea (pdf) of monetary policy. The best it can do, he thought, is to “prevent money itself from being a major source of economic disturbance.” It cannot itself achieve really good things such as lasting full employment or rapid growth. That requires the working of “those basic forces of enterprise, ingenuity, invention, hard work, and thrift”. Friedman, I think, was too optimistic about the extent to which monetary policy would facilitate the latter. But the point is that, for him, good monetary policy was mostly “a negative proposition: avoid major mistakes” – advice which, admittedly, his epigones did not wholly follow.

Yet another example comes from my day job. There’s a lot stock-pickers can do to reduce the chance of bad outcomes such as avoiding the recency bias and disposition effects, knowing the edge of one’s competence, or not trading too much. These rules reduce the chance of us losing a lot. But they don’t much help in picking the best stocks – not least because, as Hendrik Bessembinder has pointed out, these are only a tiny fraction of all the stocks on the market.

A further example is medical regulation. Demanding that drugs meet stringent safety standards before coming to market can prevent catastrophes. But this comes at the expense of delaying the roll-out of a vaccine that would allow life to return to normal. In this case, procedures that select against bad outcomes can also impede good ones.

In his new book, Economics in One Virus, Ryan Bourne claims this is true generally.  Regulations, he says, “restrict businesses and individuals’ adaptability to new conditions – they curb within-market innovation.”

Selecting against bad outcomes, then, is different from selecting for good ones. And indeed, doing the one can sometimes actually impede the other. Neoliberal management of universities, for example, has probably done a good job of selecting against academics who are idle drunks, but given the replication crisis – incentivized by the pressure to publish – it’s done less well at selecting for good work.

Which selection mechanism we should adopt depends upon context: what would have worked for the breakaway six (arguably) doesn’t work for Starmer. As Edmund Burke said: “The circumstances are what render every civil and political scheme beneficial or noxious to mankind.” If you’re running a nuclear power station your focus should be upon avoiding worst-case outcomes, whereas in creative industries mechanisms should instead select more for best-case ones: this entails trade-offs because, as Dan Ariely has shown (pdf), dishonesty and creativity are often two sides of the same coin.

My point here is simple. We must ask of any decision-making system – that is to say, any power structure – what does it select for, and what does it select against? Yes, this is true of government regulation, but it is equally true of corporate management and political processes.

Comments

[rsm]

What geniuses selected full employment and rapid growth as overall goals for society? Jobs kill people and growth kills the environment. We should select for the idea that the more you know, the less you need. Markets are predicated on keeping you in ignorance so they can sell you what you don't know you don't need.

[Nick Drew]

@ "Regulations, he says, 'restrict businesses and individuals’ adaptability to new conditions – they curb within-market innovation.'"

Well, they might tend to. But here's an important caveat from my own sector (energy market). UK regulator Ofgem can often rightly be criticised, long and hard, for all manner of failings. But for true embodiment of the above-quoted view, go to European energy regulators. They are arrogant, obtuse, close, opaque, sluggish, defensive, mulish and inflexible. They regulate, and then they step away for years at a time, irrespective of feedback & consequences.

By contrast, Ofgem is (relatively) open, consultative, responsive, non-defensive, happy to experiment and reconsider in quick iterative loops. As a result, innovation thrives in the UK energy sector: we are (e.g.) pioneering a heap of new things that will be vital for zero-carbon. I have lost count of the number of times I have been in European energy-industry forums when continentals quite genuinely say "they do this already in the UK" and "it's so much better in Britain".

There's a lesson here: regulatory processes can be bad or good, and the key is institutional / attitudinal (& thus probably cultural). Settle for nothing less than flexible & open.

[Blissex]

«We must ask of any decision-making system – that is to say, any power structure – what does it select for, and what does it select against?»

That is a very useful question, but it is also useful to ask what does (much overrated) "reality" select for or against?

The notable NN Taleb seems to think that "reality" selects for avoidance of ruin, a "satisficing" goal:

https://twitter.com/nntaleb/status/1221387129642717185
“My talk in Cyprus: without extreme paranoia, we can't survive. #ergodicity requires ignoring costs-benefits in the presence of ruin problems.”

That in an ideal world would be the role of regulation, as most power structures select for optimizing short term costs/benefits by pushing benefits to earlier and pushing costs to later.

[rsm]

Financial ruin problems are easily solved by printing more of the best money (US dollars).

Finance has figured out how to sell ergodic ensemble averages, so below-average individuals can avoid ruin by tying their bets to the ensemble average return. Stock markets go up more than down. And, the Fed can support stock prices to prevent or quickly mitigate ruinous crashes.

Costs can be put off forever, because credit eventually turns into money. See Mehrling 2011, https://ieor.columbia.edu/files/seasdepts/industrial-engineering-operations-research/pdf-files/Mehrling_P_FESeminar_Sp12-02.pdf

Note that Mehrling brushes over the fact that elasticity exceeds contraction of the money hierarchy. For example, the European debt crisis he mentions in the final paragraph was subsequently solved by more elasticity.