Stumbling and Mumbling

Spending on gambling

Polly Toynbee says:

Gambling's turnover has soared sevenfold, from £7bn in 2001 to £50bn last year. That ought to worry any government, but apparently not this one.

£50bn is indeed a shocking sum. It's more than people spent on clothing and footwear last year (£44.6bn according to table E1 of this pdf). And this should set alarm bells ringing. Maybe the £50bn is not true.
Official figures - table 09.CS of this pdf - show that spending on gambling increased from £7bn in 2001 to just £8.9bn in 2005.
Why the difference? These figures show net spending on gambling - stakes minus winnings. Polly is (I guess) referring to the gross figure - just stakes; the nearest thing I can find to her claim is a quote here from Leighton Vaughan Williams.
But does the gross figure matter? I think not. Put it this way. If you and I bet £10 on the toss of a coin, gambling turnover is £20, even though we are - over time - just swapping money; what goes around comes around. And this is what a lot of the alleged £50bn is - it's activity on betting exchanges, where people bet against each other rather than the bookies. This is mostly just City boys swapping money.
If you want an indication of the social problem caused by gambling, the best measure is what households in aggregate lose. And that's the net figure of £8.9bn.
Is this a lot? Well, it compares to spending of £28.2bn on alcohol. It's just 1.2% of total consumer spending. And it has risen 27.7% since 2001, against a rise of 19.3% in consumer spending generally.
By all means, Polly, complain that some irresponsible people are depriving their children by gambling their money away. But don't pretend it's a bigger problem than it is.

An outlier speaks

Am I a freak? The Sunday Times cites research (pdf) which suggests I am.

It says men’s IQ at 11 is strongly positively correlated with their chances of being married later in life. One standard deviation higher IQ is associated with a 35% greater chance of being married.

When I was 11, my IQ was two standard deviations higher than average; it’s since collapsed. I’m also taller than average, and have earned more than average, both of which are also correlated with marriage probabilities. On all counts, my statistical chances of having been married should be astronomically high; after all, even an average chance of being married is a high one.

But I’ve never come close to being married and don’t expect to.

This isn’t because my standards are too high*. I know it’s too much to expect a woman to have the mind of Dar Williams and the body of Kerry Katona, but most women I’ve met haven’t even had the body of Dar Williams and the mind of Kerry Katona.

Instead, I suspect my childhood IQ is a major cause of my being single. It caused me enter male-dominated environments: boy’s grammar school, a mostly-male Oxford college and the City (where early mornings militated against going out at night and meeting people). And, coming from a lower social class, I haven’t had much in common with the few women I have met. As Ruth Kelly once said to me (she was speaking generally I think): “women don’t go for men who are good-looking and intelligent. They want ones they feel comfortable with.”

The academic research suggests these IQ-related handicaps to marriage are more than offset by the tendency for high IQ to lead to higher social class. But for me they haven’t been.

However, I don’t at all feel like an outlier. I feel normalish – after all, the single geek is a common stereotype - even though the statistics say I’m not.

Which raises the question: is my feeling of being normal just one of those self-protecting subconscious mental strategies that pretty much everyone adopts?

* It’s not because I’m fat and ugly either. I'm slimmer and better-looking than this guy, who's done more than good enough for himself.

Is Big Brother racist?

Is Big Brother racist? Faria Alam says so. Everyone’s favourite contrarian says her claim is obvious. Everyone’s least favourite cat disagrees:

Oily MP George Galloway rounded on Faria, saying: "The idea that people who watch this show are bigots is preposterous. Most of the people who've won have been from minorities of one kind or another."

We can use statistics here. Excluding people who walked out, and using my judgment of ethnicity, the contestants on ordinary BB break down as follows:

BB1. 11 contestants, 2 black/Asian.

BB2. 11, 3,

BB3. 12, 3.

BB4. 13, 3.

BB5. 13, 2.

BB6. 16, 5.

Now, if there’s no prejudice of any kind, every contestant would have an equal chance of winning. But in fact every winner has been white.

The probability that this could happen by chance in a non-racist contest is simply the product of the probabilities of a white winner in each series. That is, 9/11 x 8/11 x 9/12 etc. This comes to 20%, 4 to 1.

This, I guess, is sufficiently low that it won’t shake Faria’s Bayesian prior that there is racism in BB.

Her opponents will reply that 4 to 1 shots do come in. 20% is well above the significance levels usually required to accept a hypothesis in classical statistics.

The issue, then, is not merely: are people racist? It’s: what form of statistical reasoning do we use?

But here’s a simple test. If we assume that one-quarter of future housemates will be black or Asian, then there’s a less than 5% chance that the winners of the first eleven series will all be white.

So, if we haven’t gotten a black or Asian winner in five years’ time, Faria’s theory will be looking good by the standards of conventional statistics.

Personally speaking, for the time being, I’m prepared to go with Faria. Lots of other bald men have.

Bond prices

Sean Connery's first weekly wage packet, as a milkman, was 21 shillings, in 1944. This provides a neat illustration of how much living standards have risen since then.
The milk he was delivering cost customers 9d a quart. So his 21 shillings would have bought  56 pints - though as milk was rationed, few people could get this much. Today, the wage of a full-time minimum wage worker would buy 450 pints (£202 divided by 45p - I'm ignoring taxes and tax credits). That's almost a nine-fold rise in living standards.
Put this another way. Consumer prices generally have risen 29.4 times since 1944. So, Connery's 21/- is worth only £30 in today's money - barely one-sixth of the minimum wage.
These figures suggest living standard for the ordinary worker have improved 6-9 times since the War.
They might have improved even more because there are of course countless things, such as James Bond DVDs, that weren't available in any money in 1944.
There is, though, at least one dimension in which living standards haven't improved so much. A cinema ticket cost 1/5 in 1944. So Connery could have gone to the pictures - yes pictures, none of this "movie" nonsense - 20 times. A minimum wage worker today could go only 28 times. That's an improvement of only 40%. And this is not to mention the declining quality of both films and audiences since then.
There's one caveat to these figures. I've assumed Connery was a typical milkman. But maybe his wages were lower than others because he was bad at his job - he had a licence to spill.

Riots, Islamism and Bayes theorem

The chuntering class is demonstrating its talent for fact-free theorizing over the question: what , if anything, have the French riots to do with Islam?
David Aaronovitch says: nothing. Melanie Phillips  says that's a "big lie."
In principle, though, this is a simple empirical question. We should be able to use Bayes theorem to judge the relative importance of Islam, unemployment, or anything else, in the riots.
It works like this. Consider the young residents of a suburb afflicted by a riot. Say there were 1000 of them. Let's say these are split 500-500 between Muslims and non-Muslims and that there were 200 rioters, of which 150 were Muslims and 50 non-Muslims.
Simple manipulation then tells us that 30% of  Muslims (150 of 500) rioted compared with 10% (50 of 500) of non-Muslims.
We could repeat this exercise, replacing Muslim with unemployed (or unskilled, or previous criminals or any other category)  to assess the relative strength of the  links between Islam, unemployment, or anything else, and the riots.
Of course, I've plucked these numbers out of the air just for illustration; I picked them merely to show that it's possible for there to be a link between Islam and rioting even when the majority of Muslims don't riot.
The key point, though, is that the facts are in principle knowable. What's needed is hard numbers on who the rioters were, relative to the population they were drawn from.
What's not useful is idle arm-waving. The one thing we know for certain is that we all make terrible judgments about things that are uncertain. We see correlations where we expect them, rather than where they are, we over-infer from single salient incidents, and we see evidence that corroborates our prejudice more readily than we see disconfirming evidence. The fact that a few rioters mutter something about "jihad" does not establish that the riots were caused by Islamism, any more than the fact that many other riots have nothing to do with Islam means that Islam was irrelevant in this case.
Aaronovitch and Phillips are merely demonstrating that the dead tree industry is sometimes little more than a catalogue of cognitive biases, used to entrench people's preconceptions.
Another thing: FWIW, my gut feel is with Aaronovitch. But I try not to think with my gut. And I certainly wouldn't expect anyone to pay me for doing so.

Drink, crime and clergymen

On today's Thought for the Day, John Bell railed against the evils of drink:

What do you think would happen if it were discovered that 50% of the prison population in Britain had been helped on the road to crime because of one identifiable food additive which had an adverse effect on behaviour?...So why do beer glasses not have a sign which states: 'Alcohol may lead to criminal activity of a violent nature, dangerous driving and lasting damage to the liver.'?

Mr Bell might not be abusing drink, but he's abusing statistics. Let's rejig the numbers. On one estimate, there are 9 million problem drinkers in the UK, out of an adult population of 48 million. Let's assume that all the prisoners Mr Bell describes are drawn from this 9m.  This is reasonable. After all, if drink lands you in chokey, you've a drink problem.
Now, last week there were 77,774 people in prison. On Mr Bell's figures, 38,887 of these have a drink problem and 38,887 do not.
So, we can say that the proportion of problem drinkers in prison is 38,887 in 9 million. That's 4.3 per thousand. The proportion of non-problem drinkers is 38,887 in 39m, or 1 in a 1000.
Now, you can argue around these figures, but I reckon they're ball-park OK.
So, you could say problem drinkers are more four times more likely to end up in prison than non-problem drinkers. Or you could say that 996 in 1000 stay out of prison.
I don't think either figure justifies Mr Bell's complaints.
What's more, I suspect there are many behaviours more correlated with crime than heavy drinking; Theodore Dalrymple once claimed that all criminals were smokers. And what about tattoos?
I mention this not merely to be an anorak. We are approaching the 242nd birthday (on November 10) of the original statement of Bayes theorem; it's this pdf. It seems the statistical prowess of our clergymen has not improved in this time.

Discrimination versus prejudice

Bryan Caplan posts, insightfully as ever, on statistical discrimination. There’s one issue here, though, that troubles me. Just how robust is the distinction between statistical discrimination - generalizing from personal experience to form stereotypes about the members of different ethnic groups - and narrow-minded prejudice?

The problem arises because very few people reason correctly about statistics.

One big problem is that the media almost never report statistics accurately. Take this from the Sunday Times:

Although black people make up around 10% of London's population, they are involved (sic) in a staggering 70% of the city's shooting incidents.

We’re invited to infer that blacks are hugely involved in gun crime. Let’s unbundle the numbers.

In the last 12 months, the Met reckons there were 3819 crimes involving guns. These include offences where a firearm was seen – let alone used – and where the firearm was fake or an airgun.

Let’s assume two people – a target and an aggressor - were “involved” in each incident, and that there were no repeat targets or aggressors*. That gives us a total of 7638 people involved in gun crime. On the Sunday Times numbers, 5347 of these were black and 2291 non-black.

Now, there are 7,421,228 people in London. On the Sunday Times figures, 742,123 are black and 6,679,105 non-black.

So, the proportion of blacks “involved” in gun crime in the last 12 months is 5347 out of 742,123 – 721 per 100,000.

The proportion of non-blacks involved is 2291 in 6,679,105 – 34 per 100,000.

So, blacks are indeed more likely to be “involved” in gun crime – 21 times more likely in dead-tree-speak, or by 687 per 100,000 in more reasonable terms.

Statistical discrimination tells us to associate blacks, more than non-blacks, with gun crime.

But is this a big difference? In another sense, blacks and non-blacks have an enormous amount in common, in that neither are likely to see either end of a gun. 99,279 out of 100,000 blacks were not “involved” in gun crime last year. And 99,966 non-blacks were not “involved.”

But then, the line “blacks overwhelmingly unlikely to be involved in gun crime” isn’t much of a headline.

We shouldn’t, though, blame only the media for encouraging excessive statistical discrimination. There are other biases involved

1. The salience heuristic. People draw stronger inferences than they should from prominent information. So, if a black man shoots someone, they infer that blacks are more likely to shoot people. But they fail to draw inferences from the millions of everyday ordinary acts of black people. This problem becomes especially severe if groups are segregated from each other. Then, acts of violence are likely to be disproportionately the only times that whites see blacks, or Jews see Arabs.

2. Framing effects. Say a 6’4” black man who works as doorman shoots someone. People will call this a “black crime,” and draw inferences. But why not call it a “tall crime”, and infer that tall people are a menace? Or a "doorman crime"? There are fewer tall people or doormen than black ones, so – on this datum – the proportion of tall people or doormen who are criminals is higher than the proportion of blacks. Why focus on colour, unless it is because there's some back-story to warrant doing so?

3. Conservatism bias. Once people have a view, they stick to it to a greater extent that they should. The man who believes blacks are likely to be muggers ignores the huge majority of blacks who don’t mug him.

In practice, then, many instances of statistical discrimination are intellectually wrong. In particular, they exaggerate the differences between blacks and whites.

Which raises the question. Is there a point at which intellectual error becomes a moral error?

* this is unrealistic, but it probably biases the figures against my point. And I’ll take the Sunday Times figures on blacks at face value, rather than get into issues of racial categorization.

Valuing education

The Times' dead tree version reprints its VE Day issue. Its front page (victory in Europe was an inside-page story) carries an advert for a professor of geography at the LSE. Salary – a minimum of £1400 a year. That’s £122,000 in today’s wages. A lectureship in colonial history at the University of Edinburgh carried a stipend of around £600 - £52,233 today.

The 1940s was the age of austerity, remember.

And house prices? A 10-bedroom house near Chelmsford – with “stabling, out-buildings, 2 cottages, well-timbered gardens and grounds of nearly 5 acres” – cost £7500. That’s five-and-a-half years’ pay for our LSE professor. Something similar today would cost around £2 million. Had LSE professors’ pay kept pace with house prices, they’d be earning almost £400,000 a year.

“There is no greater responsibility than teaching the next generation.” – Labour manifesto.

Some unpleasant comparisons

One of the safer bets for tomorrow's drone-fest Budget is that Gordon Brown will spend some time crowing about the UK's impressive macroeconomic performance. Certainly, today's news about the Italian economy suggests that the UK is in good shape relative to some of our big European trading partners.
However, in one respect, the UK has done worse than the euro zone under Mr Brown's Chancellorship. Here are figures on the growth in manufacturing output since he became Chancellor in May 1997:
Germany: +18.6 per cent.
Spain: + 17.6 per cent.
Euro zone: +15.6 per cent.
France: +14.2 per cent.
Netherlands: +6.4 per cent.
UK: +2.1 per cent.
Italy: -2.1 per cent.
(Comparisons are  the latest three months relative to May 1997 - they come from Datastream so there's no link, I'm afraid) .
So, the UK's manufacturing sector under Mr Brown has been one of the worst performers among large European economies.
Can this really be all due to the strength of the pound for much of this period? Or does it show that Mr Brown has worsened UK competitiveness in some respects? And how much does manufacturing matter anyway?

Jordan, Bayes and Friedman

On seeing this story about Jordan, one's thoughts turn naturally to Bayes' theorem. It shows that the plaggy hootered minger might have been right to take nine pregnancy tests before deciding she was pregnant.
The theorem gives us a way of rationally updating our beliefs in the light of new evidence. In the context of pregnancy tests, it works like this.
First, you multiply your prior probability that you're pregnant by the true positive rate on a pregnancy test.  For the  sake of argument, let's say Jordan thought there was just a 10 per cent chance of her being pregnant, before taking any tests. And let's say the true positive rate is 95 per cent; suppliers claim their kits are 97-99 per cent accurate in laboratory conditions, but I guess the Jordan gaff is not a laboratory. This gives us 0.095.
Then, you multiply  the probability that you are not pregnant (0.9 in this case) by  the false positive rate on pregnancy tests. Let's assume this  rate  is 5 per cent. This gives us  0.9 x  0.05 =  0.045.
Then, you add this number to  our first calculation. That gives  0.095 + 0.045 =  0.14.
Then you divide our first calculation into this number.  This is 0.095 divided by  0.14  =  67.9 per cent.
This is would be Jordan's estimate of her probability of being pregnant, given one test.
This is clearly a low probability. So it would have been entirely sensible to take more tests. Just how many would depend upon the correlation between one test and another; if the test results (or their interpretation) were uncorrelated, a second positive test would raise the probability of being pregnant to 97 per cent. If they were correlated, the probability would be lower.
However, it's easy(ish) to tweak these numbers to justify taking nine tests. Maybe Jordan's prior was less than 10 per cent. Maybe test results were correlated; false results, I believe, are more likely if you're taking some types of medication. Or maybe I'm over-estimating the reliability of the results under the conditions Jordan took them.
So, Jordan might have been behaving as a good Bayesian.
But of course, few of us believe she and Peter sat down and computed Bayes theorem before deciding to do nine tests.
In other words, although she did not behave rationally, she did behave as if she were rational.
And this is where  Milton Friedman  comes in. In his essay, the Methodology of Positive Economics (an excerpt of which is here), he said that economic theories should be judged by their predictive power, not their descriptive accuracy. So, he said, what mattered is that businessmen behave as if they are minimizing a cost function, or that consumers behave as if  they are mazimizing utility, even though no-one actually sits down and solves the equations.
So, isn't Jordan a bit of evidence that the basic postulate of neo-classical economics is valid, in the Friedmanite sense? People might not behave rationally, but even dim ones can behave as if they were rational.

Update: For the benefit of American readers, "plaggy" is a derogatory word for plastic, usually of inferior quality - as in "plaggy gangster", a small-time criminal who thinks he's a big cheese. "Hootered" derives from this fine company. And a minger is a munter or moose. Americans might not possess the phrase "plaggy hootered minger" - though I'm surprised Larry Summers hasn't coined it - but they have the things in abundance. Hope this helps.