Cognitive biases in football

Cognitive biases are everywhere - in stock markets, boardrooms (pdf), politics and even songs.It should, then, be no surprise that they also affect football punditry.
I'm thinking here of four biases:
1.Misperceiving randomness. There's always a chance that a team will suffer from bad luck: its opponents' finishing will be unusually brilliant; refs and linesmen will make bad decisions* against it; or usually good players will make bad errors. These random events will sometimes bunch together - that's the nature of randomness. If they come in one game, the team will get a hiding. If they come over a few successive games, it could suffer a losing streak.
But the pundits won't appreciate this. Instead, we'll hear about the club losing form, or being in crisis or decline. But in fact, what might have happened is just an "accident."
2. Hindsight bias. Results colour our attitude to teams' performance. If a team wins, we naturally think it played well and this can distract us from its shortcomings. For example, Manyoo's victory over Chelsea covered up the fact that they conceded 20 shots. Had things gone a little differently - better finishing by Chelsea or two of United's goals being ruled offside, this would have led the pundits to question the quality of their young defenders.As it is, they escape scrutiny from the media (though not, I suspect, Sir Alex).
3. The availability effect. Some aspects of the game are more noticeable, mentally available, than others, and so they attract an attention disproportionate to their importance.
The stupidest example of this is when retards criticise managers for not being more passionate on the touchline. But this is just the 2% of a managers' working week that is the most salient.
A more subtle form is that set pieces are more salient than many aspects of open play, such as the transition from possession to non-possession. This means that if a team is poor at defending set pieces, it'll get a reputation for bad defending even if its overall defence is good.You'd never guess from all the howling that,last season, Arsenal had the fourth-best defensive record in the league despite: having no goalkeeper for half the season; missing their best defender for most of it; not parking the bus in front of the goal;and being the victim of some appalling officiating.
4. Confirmation bias. Once people have an idea, they find evidence to corroborate it.So, having gotten the idea that Arsenal can't defend, they overweight evidence for this, an underweight evidence to the contrary. This is all the more likely to happen if it combines with point 1.
The counterpart of this bias is that people don't admit they were wrong.12 months ago, everyone said Arsenal needed to buy a new keeper. Wenger thought differently. And when Szczesny made that save against Udinese, did Wenger's critics recant and say that, maybe, they are wrong on other things as well? No.
Now, although fans and pundits commit these errors, coaches rarely do. As Arsene says, "We must not completely believe what people say when it goes well or badly." This isn't because they are better educated in statistics than their critics - though Arsene is - but rather because rationality often arises from practice, so those better practiced in football make fewer intellectual errors about it.This means that coaches don't over-react to good or bad streaks. They "take positives" from a defeat not (just) out of the self-serving bias but because there often are such positives. And they sometimes - though rarely in public - describe a winning performance as "disgraceful."
And yet, despite all this, fans and journalists continue to criticize coaches who have vastly more knowledge and expertise than they do. Which is, surely, an example of the Dunning-Kruger effect.
* I'm assuming refs' bad decisions are random. This is not obvious.

Comments

[Tom Addison]

Great post, got to offer my two cents on it.

I suppose a lot of it comes down to the fact that football fans need to come up with stories to explain what happened, which obviously ignores the random events that happened on the way. Man United winning the CL in 1999? Well, it was on Sir Matt Busby's 90th birthday, therefore it was inevitable (despite us being a few minutes way from defeat, and deservedly so). Man United winning the CL in 2008? Well, it was the 50th anniversary of the Munich disaster, therefore it was inevitable (despite us being one John Terry penalty away from defeat).

What also annoys me is the "small team bias" (my term!) displayed by pundits and commentators. For example, Arsenal or United or whoever may have 20 shots on goal and 70% possession against a Norwich or a Burnley in the first half, yet the commentators will talk about how "bravley" the smaller team is defending (even though it may just be luck), and say that they "deserve" to be level at half-time.

The number of shots we conceded against Chelsea (and for the whole season so far) is a great point, probably explains why I felt so weird at full-time in that game. Many said before the game that United hadn't been tested properly this season, that the likes of Anderson were only playing well because they'd had an easy ride so far (which I'd agree with), and the second half against Chelsea confirmed that. It was our first real test. We've got away with it though.

Regarding point 3, Michael Carrick is often a victim of this. Your simpler football fan will watch what's happening on the ball, whereas the football fan that is more appreciative of the game will watch what's happening off the ball. Surely preventing a pass from happening in the first place by being in the correct defensive position (as Carrick does) is as good as lunging in and having to tackle someone (as Anderson does?) It appears not.

"A more subtle form is that set pieces are more salient than many aspects of open play, such as the transition from possession to non-possession." You read Gab Marcotti's article in The Times today, didn't you?

Aaah, confirmation bias. Again, Carrick is a victim, as is Berbatov, and currently De Gea is as well. By the logic of the people that wrote him off, when he made that penalty save against Arsenal, that should have made him the best keeper in the world.

You should write a book on this.

[ejh]

"What also annoys me is the "small team bias" (my term!) displayed by pundits and commentators"

Do me a favour.

[Tom Addison]

I shall keep a log of it from hereon. If I'm wrong, I guess I owe you a Coke.

[ejh]

"And yet, despite all this, fans and journalists continue to criticize coaches who have vastly more knowledge and expertise than they do. Which is, surely, an example of the Dunning-Kruger effect."

Right, this won't do at all, on several grounds.

a. Fans, and journalists, will criticise, and should criticise, because that's their role in the game, and because that's a function of the passion which they feel for it. If you go to the cinema a lot, you criticise, even though you could barely press the button on a camera. If you read a lot of novels, you criticise, even if you could barely write a coherent sentence yourself. No criticism, no following: it's really as straightforward as that. A crowd that will not boo will not cheer, not would that cheering be meaningful if it did.

b. Fans, and journalists, are often extremely knowledgeable about the game. Many years of watching, not to matter all sorts of attendant reading, to very often have the effect of imparting knowledge and experience to the heads of the people who do so. There are many fools in football crowds, and in the football pages, and there are many people in both locations who are neither. As knowledgeable as lifetime practitioners? Probably not, but that doesn't mean that the professionals are always right, while the amateurs are always wrong. (It does mean the professionals have to live by their decisions, and have to have their decisions tested in practice. These are important points, but they're not the only ones.) Nor as the criticisms of fans and journalists necessarily, or even generally, stupid ones.

c. There are biases of other kinds too, notably the one in which professionals in any field tend to back up one another in public regardless of their private opinions. This may well lead football professionals to overrate the job that a manager is doing whereas fans and journalists, having no such institutional bias, can sometimes see more clearly. The tendency to write off the opinions of fans, in particular, is itself a very heavy institutional bias in football.

In re: Arsenal and Wenger, for instance - and I write as someone who supports neither Arsenal nor any of their rivals, and who rather admires Wenger - it might be useful to ponder that Arsenal fans (who themselves have of course a variety of opinion on Wenger) have seen a notable though not drastic decline in their team's powers over half a decade, and a serious decline in that team's capabilities over less than a season, in so far as what looked to be a team of title contenders quite a way into last season turned rapidly into something of a shambles.

How true that is, and what lesson to derive from it, is of course difficult to judge. But it would be a better starting point, recognising that there is a problem and that the disgruntled are complaining about something real, the nothing-to-see-here line, still less the "they must be mad" line which characterised a lot of commentary about Arsenal complainers, until they started losing a lot, and heavily at that. Perhaps the fans did know something, after all.

Shorter me: I've called for a lot of managers' heads in my time. And you know what? Sometimes I've been right.

[A1]

Terrible isn't it. These experienced managers getting criticised.

[Student]

Its posts like this (and the pop music one) that keep me coming back to this blog again and again.

I like your thinking Dillows.

As an Arsenal fan myself I think all this Wenger's-got-to-go speaks says more about the critics - what would it change? Arsenal are hardly in danger of relegation this early on in the season, and yet the talk of collapse persists. Id say this result has rather low degrees of freedom.

(Your links on the right to your IC pages are broken btw)

[Little Tripoli]

Love your analysis - and, given Arsenal's current travails, I wondered how long something of the sort might appear...

Just out of interest, do you read 7am kick off? He's the only one of the Arsenal blogs that (to my mind) came close to offering a cogent dissection of the United defeat, and it closely resembles your thinking:

http://www.7amkickoff.com/2011/hitting-the-bar-shelter-from-the-storm/

[Pb204.blogspot.com]

Randomness has a big effect on football results. To a good approximation, a football match can be simulated using a large number of time slices, in each of which each team has a given probability of scoring a goal. The effect is that the number of goals each team scores in a match is a sample from a Poisson distribution. It's characteristic of a Poisson distribution that the variance is equal to the mean, so that for sports, like football, with low mean scores, the standard deviation is large.

For example, in a match in which on average team A is expected to beat team B 2-1, team A will in practice win about 60% of the time, and team B will win about 18% of the time.

[CS Clark]

With regard to the Dunning-Kruger effect, while fans and pundits may overestimate their ability to answer questions such as 'Are Arsenal rubbish?' and 'Should Wenger feck off?', so too are managers and players likely to overestimate their ability to answer questions posed by fans such as 'Is it worth my time and money to watch this?'

[George Hallam]

"Cognitive biases are everywhere .. I'm thinking here of four biases"

Excellent. You can do it for football. Now apply this to your attitude to the Soviet Union.

[Bulk Email]

Great article, this is could be applied to sooooo many aspects of life.

[skidmarx]

' "they must be mad" line which characterised a lot of commentary about Arsenal complainers'

Apart from a brief period before the Champions League Final against Barcelona, I can't think of a time when most commentary wasn't damning Wenger with faint praise and suggesting that it was understandable if fans wanted something more than pretty football.

Kevin Pullein of the Racing Post might know if there is measurable ref bias. He certainly had an interesting piece last week on how Keynes principles for investment, firstly never go with the crowd, are transferable to football betting. John Foot's Calcio had something to say about whether Italian refs are submissive to the big teams.

Wondering recently about why Arsenal seem to have stopped regularly knocking 3 or 4 goals past weak teams as it did in the days on Henry even when he didn't play, I thought one possible explanation might be the partial abandonment of the strict passing game in favour of more crossing and heading might be to blame, of course this is the opposite of the conventional belief about what might be wrong.

[Rumbold]

Interesting piece, but why the casual abuse of disabled people?

[Paul Sagar]

None of this changes the fact that Wenger is a nobhead.

[Torquil Macneil]

What also annoys me is the "small team bias" (my term!) displayed by pundits and commentators

Good god! Somone who thinks the media don't pay enough favourable attention to Manchester United?! Surely that deserves a psychological category all its own.

[Igor Belanov]

I would agree with ejh, with the addition that fans' expectations have radically changed with the massive financial/power disparities, and this affects their judgement.

For a team like Arsenal, after their earlier success under Wenger, there is an assumption among the fans, and fuelled by the media, that they should be challenging closely for league and cups every season. It was made a big issue after Arsenal's surprise League Cup Final defeat last season that they hadn't won a trophy since 2005- not that long ago really. Now they have experienced such a poor start to the season I expect fans are afraid that they will miss out on Champions' League money and slip further behind the two Manchester clubs, Chelsea, and even more frightening for them, Spurs.

The opposite effect is evident at the other end of the Premier League. A few weeks in the relegation zone make everyone very twitchy, but fans of these clubs have been encouraged by the media to treat 17th place every season as a success. This affects attitudes to cups- many managers and fans of relegation-threatened teams actually seem to be relieved to be eliminated from them. After Birmingham followed Carling Cup triumph with poor league form and ultimate relegation there were many commentators who thought that it was the cup run that had caused it!

[Silence Is Golden]

I think this sums it up:

http://xkcd.com/904/

[Lee T]

There is a bias towards criticism in football, which I think is unavoidable.

At any one time only a minority of clubs will be achieving what their own supporters want them to achieve.. so their own fans will be critical. Few football fans seem able to accept the place their club holds in the natural order of things (see also many chairmen.. the sacking of Sean O'Driscoll yesterday is a disgrace).

Pundits, then, need to reflect the views of their audience or they will lose it. Further, when they discuss Club A they are discussing a club which most people either have no interest in, or actively dislike. so, again, being critical is likely to be more popular than being praiseworthy. Both are much more popular than being balanced or rational. The BBC now employ one R Savage as a pundit and phone-in host. This is in addition to the epic eejit that is Alan Green. These are not appointments made by a broadcaster seeking intelligent and even-handed analysis.