Karen Matthews & cognitive bias

If there’s one lesson to be learned from the Shannon Matthews affair, it’s that there are no lessons to be learned.
Put it this way. Human nature being as it is, some welfare mums are criminals, some are heroes struggling under terrible circumstances, and many are just ordinary folk. There’s a distribution of qualities.
But we can infer nothing at all about the shape of any distribution, or its average or standard deviation, from just seeing one extreme data-point.
So, how come so many people think they can “learn lessons”?
Cognitive biases, that’s how.
One bias is the availability heuristic. It’s natural to think that events that spring readily to mind are common, and representative of the wider truth. So, when asked the question: “are welfare mums idle criminal scroungers?” Karen Matthews springs to mind, and people think “yes, they are.”
Now, very often this sort of bias is correct. For example, if we want to consider whether Arsenal are a better team than West Ham, we find it easier to recall Arsenal victories  than Hammers’ ones, and so come to the right answer.
Where the bias goes wrong is when it’s conjoined with another error - the ignorance of selection biases. Karen Matthews got media coverage precisely because she was atypical; “man shoots dog” is not news, but “dog shoots man” is. What we cannot know from the case alone is just how atypical she is.
Here, though, another bias comes in - the confirmation bias. We see what we want to see. So, David Cameron sees Ms Matthews as evidence of a “broken society”, Bea Campbell as evidence of “impoverished” state childcare services,  Norman Bettison as an exception to a community with “moral spirit” , James Bartholomew as a creation of a demoralizing welfare state, and Mr Eugenides as evidence that the commentariat are full of bull.
They may be right or wrong.  But none are “learning lessons.” They are merely having their prior beliefs confirmed.
Stan Moss shows us a neat example of this. He points out that people of all classes commit horrible crimes, but none of us infer that working in banks or serving in the RAF has a corrupting effect. We don’t do so because we don’t have a prior belief that they do so.
So, maybe there are no lessons to be learned here, except that political debate is riddled with cognitive errors.
But then, in saying this, mightn’t I too be committing the confirmation bias, and seeing what I want to see?

Comments

[Matthew Cain]

Matthew Taylor's right that this public moralising is on the increase: http://www.matthewtaylorsblog.com/socialbrain/sharon-shoesmith-and-karen-matthews-not-totally-black-and-white/

[Mike Woodhouse]

"mightn’t I too be committing the confirmation bias"

Doesn't mean you aren't reaching the right conclusion, just that you might be reaching it for the wrong reasons. Of course, I agree pretty much 100% with your analysis on this matter, which leaves me vulnerable to confirmation bias myself. Oh dear...

[Anthony T]

I don't think you're affected by confirmation bias as you're not confirming or rejecting any of the things people have concluded from this case.

Ultimately we need to seek evidence free of sample bias to draw conclusions. Statistics may be viewed cynically but they can still be more robust than anecdotes!

[Nick]

The various police blogs that I read seem to think Matthews's behaviour is extreme but indicative of more moderate forms of the same mentality. Of course, they deal with the worst people every day and those that are blogging might be likely to be more disgruntled. Nevertheless, we can't say without further information whether we have a cognitive bias or not. But we know crime,unemployment, drug use and child abuse are correlated with a single parent family structure - so without jumping to causation conclusions - we ought to say something is going wrong, and that something is not purely to do with material prosperity.

[David Heigham]

The lesson is dead simple; and simple economics. When you create perverse incentives, or even the appearence of perverse incentives, you can expect them to do damage in ways that you did not expect as well as the ones that were foreseen.

[ajay]

"...none of us infer that working in banks... has a corrupting effect."

Oh, don't we, by jove?

[Bob B]

We first look to see where the head offices of the failing banks were located and draw the inevitable appropriate conclusions.

[Nick Hardy]

How would you fit 9/11 within this framework, then?

[ajay]

Bob b: very good...

Nick: fits very well; it's an availability error. People got terrified of dying in a terrorist attack because it got far more publicity than (say) 3000 people dying in car crashes over the next two months.

[Bob B]

As for perverse incentives and to make comparisons with Karen Matthews, try Robert Peston on: The Greed Game:
http://video.google.co.uk/videoplay?docid=-4001834874264918973

So much for John Redwood's claim that interest rates were kept too high for too long.

"almost three years after stepping down as chairman of the Federal Reserve, a humbled Mr. Greenspan admitted that he had put too much faith in the self-correcting power of free markets and had failed to anticipate the self-destructive power of wanton mortgage lending.

"'Those of us who have looked to the self-interest of lending institutions to protect shareholders’ equity, myself included, are in a state of shocked disbelief,' he told the House Committee on Oversight and Government Reform."
http://www.nytimes.com/2008/10/24/business/economy/24panel.html?ref=todayspaper

No wonder some pols don't want any policy interventions - or new regulations - that could spoil the game and restrict the returns. Better to turn the spotlight on the likes of Karen Matthews to distract attention.

No surprise there.

[anon]

"So, maybe there are no lessons to be learned here, except that political debate is riddled with cognitive errors."

Can you call it a lesson, when everyone knew it already?

[Iain]

You're a die hard empiricist (positivist) then. As a die hard Constructivist, I feel you totally misunderstand the role of examples to provide 'truth'. You are under the misconception that the truth (perhaps a Black Swan) will arise by looking at the typical, through your random-sampling. But you are wrong. The truth, in so much as you can find such a contested construct, is found in the extreme, atypical example. Look at how humans learn-- they need the basic deductive tools to get to beginner stage but they need depth to get any higher. Extreme examples provide that depth. Extreme examples are how people actually learn. Think how Galileo 'proved' Aristotle wrong. Did he widely sample looking at the typical. No, he used on extreme case. Scientists pay lip service to the notion of sampling but don't actually apply it in analysing humans.

This is such a schoolboy error. Please read:
"Five misunderstandings about case study research", Qualitative Inquiry, Vol 12, No 2, April 2006, 219-245

[Bob B]

"Extreme examples provide that depth"

Quite so. It's become fashionable in recent years to remake old, popular movies in a modern ambience but I somehow doubt that wonderful ol' classic: I'm all right, Jack, will make the growing list of remakes:
http://uk.youtube.com/watch?v=ZAii8tPrwtQ

For a start, I think they'd have a bit of difficulty in raising the finance even with a new all star cast and featuring David Cameron and George Osborne in cameo performances. Besides, trade unionists are hardly the villains nowadays compared with bankers, financial traders and their political friends.

As reported by an impeccable source, 19 of the 29 members of the Conservative shadow cabinet are millionaires:
http://conservativehome.blogs.com/torydiary/2008/07/does-it-matter.html