Two different comments on different subjects reveal a common error in thinking about social affairs. First, in response to my claim that much of the gender pay gap is due to women having children, Toto says: “you didn't consult any childless women before writing this, did you?” You’re damn right, I didn’t.
Second, a commenter on a post by DK says:
The obvious one is small sample bias. What we see ourselves is only an infinitesimally small fraction of the 60 million people in our country. Drawing inferences from such teeny samples is fraught with problems, for example:
1. Selection bias. The people we see are rarely a representative sample. For example, the childless women I know earn much more than the national average.
This can lead us into horrible errors. For example, journalists see colleagues and university friends earning good money and infer than average incomes across the country are high, which leads to the middle England error.
2. Overconfidence. People under-estimate error margins. So they fail to appreciate the huge sampling error of a small amount of evidence.
3. Out-group homogeneity bias. We tend to believe that “others” are more alike than they really are. So, from seeing one or two immigrants act in a particular way, it’s easy to infer that many do.
4. Confirmation bias. Having made the above errors, we compound them by subsequently over-weighting the significance of apparently corroborative evidence.
These problems mean we cannot rely upon our own eyes. We need large-scale studies. And in these two cases, these happen to disconfirm the commenters. For example Bob Rowthorn estimates that the net fiscal cost of immigration to the UK is roughly zero. Yes, some immigrants require the tax-payer to stump up more money. But other immigrants provide these taxes. It’s a wash. And this paper (pdf) says that some 36% of the pay gap is due to women’s different life-time working patterns. This one (pdf) finds that 10 percentage points of the 25 percentage points gap between women and men’s pay after 10 years is due to differences in human capital accumulation. And National Statistics - which don’t adjust for other things - show (pdf) that the full-time gender pay gap is actually negative for single people, but increases sharply with the number of children. I think all this is consistent with my use of the word “much”: not “all”, or “most” - just “much”.
Which brings me to a problem. Most people, I suspect, don’t base their beliefs about society upon rigorous social studies but rely instead upon their own eyes. Does this matter?
The wisdom of crowds hypothesis says not. For everyone who sees an outlying woman earn more than a man, others see women earning less. Across everyone, the crowds get it right.
Or do they? Let’s take immigration. Maybe, if you walk into a doctor’s surgery you will see lots of immigrants (not just the doctor himself!) and infer they are a drain on the tax-payer. In principle, this inference could be offset by people walking into factories or farms and seeing hardworking immigrants who pay tax. But this doesn’t happen. Many more people wander into surgeries than into factories. The availability heuristic therefore leads people to over-estimate the extent to which immigrants are a burden.
Perhaps, therefore, the evidence of our own eyes can be systematically misleading, even when aggregated over everyone. Which in turn implies that the opinion of the majority might be wrong, even if it is not affected by the trash media.
Second, a commenter on a post by DK says:
Both commenters make the same mistake - they think we can trust the evidence of our own eyes. We can’t. Such evidence is subject to horrible distortions.Try visiting an area that has experienced high levels of immigration and telling the teachers, doctors and police there that the sudden influx of a large number of non-English speakers is not an event that requires substantial additional resources.
The obvious one is small sample bias. What we see ourselves is only an infinitesimally small fraction of the 60 million people in our country. Drawing inferences from such teeny samples is fraught with problems, for example:
1. Selection bias. The people we see are rarely a representative sample. For example, the childless women I know earn much more than the national average.
This can lead us into horrible errors. For example, journalists see colleagues and university friends earning good money and infer than average incomes across the country are high, which leads to the middle England error.
2. Overconfidence. People under-estimate error margins. So they fail to appreciate the huge sampling error of a small amount of evidence.
3. Out-group homogeneity bias. We tend to believe that “others” are more alike than they really are. So, from seeing one or two immigrants act in a particular way, it’s easy to infer that many do.
4. Confirmation bias. Having made the above errors, we compound them by subsequently over-weighting the significance of apparently corroborative evidence.
These problems mean we cannot rely upon our own eyes. We need large-scale studies. And in these two cases, these happen to disconfirm the commenters. For example Bob Rowthorn estimates that the net fiscal cost of immigration to the UK is roughly zero. Yes, some immigrants require the tax-payer to stump up more money. But other immigrants provide these taxes. It’s a wash. And this paper (pdf) says that some 36% of the pay gap is due to women’s different life-time working patterns. This one (pdf) finds that 10 percentage points of the 25 percentage points gap between women and men’s pay after 10 years is due to differences in human capital accumulation. And National Statistics - which don’t adjust for other things - show (pdf) that the full-time gender pay gap is actually negative for single people, but increases sharply with the number of children. I think all this is consistent with my use of the word “much”: not “all”, or “most” - just “much”.
Which brings me to a problem. Most people, I suspect, don’t base their beliefs about society upon rigorous social studies but rely instead upon their own eyes. Does this matter?
The wisdom of crowds hypothesis says not. For everyone who sees an outlying woman earn more than a man, others see women earning less. Across everyone, the crowds get it right.
Or do they? Let’s take immigration. Maybe, if you walk into a doctor’s surgery you will see lots of immigrants (not just the doctor himself!) and infer they are a drain on the tax-payer. In principle, this inference could be offset by people walking into factories or farms and seeing hardworking immigrants who pay tax. But this doesn’t happen. Many more people wander into surgeries than into factories. The availability heuristic therefore leads people to over-estimate the extent to which immigrants are a burden.
Perhaps, therefore, the evidence of our own eyes can be systematically misleading, even when aggregated over everyone. Which in turn implies that the opinion of the majority might be wrong, even if it is not affected by the trash media.
The wisdom of crowds doesn't provide an escape here. Those people who complain about immigration feel strongly about it; and so tend to be loud. Those who don't care one way or the other ('it's a wash') tend not to say much. So public debate is dominated by the obsessives.
Next, the availability heuristic leads us to think that if there is so much talk about immigration being a problem, then perhaps it is.
Wisdom of crowds needs independent decisions; mass media tends to make the wisdom of crowds become the stupidity of mobs (This isn't a Sun or Mail reader's problem alone; the guardian-reader is every bit a part of their own mob).
Posted by: William | October 27, 2009 at 01:19 PM
As Willliam implies, there is the 'Spartan factor' at work as well as all those you cite. In ancient Sparta, votes in the Assembly were decided by which side shouted loudest. Under Blair and Brown, an increasing number of decisions to bring forward legislation seem to have been determined by listening to the volume and stridency of media shouting.
Posted by: Diversity | October 27, 2009 at 05:18 PM
Both commenters make the same mistake - they think we can trust the evidence of our own eyes. We can’t. Such evidence is subject to horrible distortions.
So 4000 Muslims protesting in the streets cannot be trusted because our eyes saw it?
Posted by: jameshigham | October 27, 2009 at 07:57 PM
"So 4000 Muslims protesting in the streets cannot be trusted because our eyes saw it?"
Does this sentence even make sense to its author? What can't be trusted? The Muslims? The streets? The number? The fact they were protesting? What can't we trust them on?
Posted by: Alex | October 28, 2009 at 08:25 AM
tricky ... what does "do not believe your own eyes" tell us to think, for instance, about this:
http://fatmanonakeyboard.blogspot.com/2009/10/managing-mail.html
(from your Top Blogging links)
Is Roy Mayall right to believe his own eyes? Do we believe that Royal Mail managers are deliberately falsifying the statistics to create the impression mail volumes are falling?
Posted by: Luis Enrique | October 28, 2009 at 10:52 AM
Luis - that rather depends on the sample size, doesn't it?
Posted by: Neil | October 28, 2009 at 12:12 PM
The problem here is quite clear from both comedy troll James "paedophile gay mafia" Higham (not joking: http://stumblingandmumbling.typepad.com/stumbling_and_mumbling/2009/10/marriage-prejudice.html?cid=6a00d83451cbef69e20120a5f1e974970b#comment-6a00d83451cbef69e20120a5f1e974970b) and Luis Enrique's posts. Rejecting anecdotes in favour of data isn't a question of partisanship.
Both the person who saw x thousand Muslims (or whatever Higham is on about), and Roy Mayall are human beings and therefore subject to the standard biases, staring down their own personal straws at the landscape and imagining they are telescopes. Mayall's claim sounds a lot like denialism to me, but frankly I dunno.
Posted by: Alex | October 28, 2009 at 01:39 PM
Alex,
well ... quite. But Roy Mayall is an example of somebody calling reported data into doubt, using anecdote to suggest that what data we have is based on falsified management estimates. Of course the only way we have to find out the truth is to conduct an independent, well-designed data gathering exercise. I presume no such data exists, but Roy's anecdote might provide us with reason to gather it. How else would we have reason to doubt management, if not from people telling us it does fit with what they've seen with their own eyes?
But on second thought, this isn't such an interesting point; Chris is not arguing that we should never give any weight to direct observation, only that we shouldn't regard it as sufficient basis for informing our beliefs.
Posted by: Luis Enrique | October 28, 2009 at 03:40 PM
Of course, hardly anyone could be better placed to check those particular data than the CWU, which has a presence in every sorting office and actually could have its members sample them. It might be quite an interesting form of protest. (But then, if it really is declining, I suspect management and labour would be united in denial.)
Posted by: Alex | October 29, 2009 at 08:25 AM
Yes, it's really not clear to me why Royal Mail management would desire to report lower-than-true volume numbers - usually managers are busy falsifying numbers in the other direction. If lying about volumes helps evil managers in their scheming to make workers redundant, why don't other firms do it? Perhaps there is something special about the Royal Mail that means it's in the interests of management to pretend the business is in a worse state than it is.
(the CWU might be well placed to collect the data, but they'd be advised to have some independent involvement, if they want to be trusted)
Posted by: Luis Enrique | October 29, 2009 at 09:32 AM
Well, in order to create a permanent crisis narrative that supports their self-serving lobbying for privatisation...
Posted by: Alex | October 29, 2009 at 10:54 AM
ah! I was thinking that didn't make sense, because when you float a company, you usually want to make it look as good as possible, so that you can raise as much money as possible, and managers are usually given more lolly the higher the share price they achieve is ... so usually you'd avoid the impression of a crisis at all cost ... but in this case the managers may not care how much money is raised by privatization. This is the well worn City practice of "kitchen-sinking", isn't it - get it privatized for a low a price as possible, make the performance look as bad as possible, then state a miraculous recovery, get praised as management geniuses, watch the share price rocket, and quids in.
If this is what's going on, somebody needs to change the terms of managements' contract so that they get paid for floating the business as at high a price as possible, and have new management hired post-privatization.
Posted by: Luis Enrique | October 29, 2009 at 11:28 AM
(Chris, one for the tips section of your day job there: Buy Post Office shares upon IPO)
Posted by: Luis Enrique | October 29, 2009 at 11:43 AM
To whom it may concern,
I was wondering if there are any procedures to enable a patient to
enhance or change their eye shape (to make it bigger, or have an
upward slant to achieve a sexy look).
Also, I would like to know what can be done for 'asymmetrical eyes'
are there any correcting procedures???
Posted by: propecia | April 26, 2010 at 07:29 PM