A campaign for social science

Listening to this (about 14 mins in) and this, the idea occurred to me that if I weren’t so lazy, I would set up a campaign for social science.
This would be a little different from More or Less or Bad Science, great as they are. Very often, they deal with the abuse of statistics. I’m thinking not just of this, but of fact-free hand-waving. What my campaign would do is insist upon more scientific standards of discussion of social affairs. This would entail three things:
1. Be explicit about mechanisms. What is the mechanism through which a Pakistani background might predispose a man towards sex crime? What’s the mechanism that links prison to re-offending?
Immediately, we run into a problem. In theory, there are almost always opposing mechanisms; think of income and substitution effects. For example, prisons might frighten criminals away from crime, or association with fellow criminals might teach them new criminal techniques. Islamic culture might inculcate strong norms against sex crimes, or it might instead cause a contempt for “easy” white women.
2. Given that mechanisms are, in theory, offsetting, we need evidence. This does not mean anecdotes. Pretty much any mechanism applies to someone, so anecdotes are useless.
Instead, we need statistics. Are Pakistanis more likely to commit sex crimes than others? (Kudos to Unity here). Are imprisoned criminals more likely to return to crime than comparable criminals who get non-custodial sentences?
In this context, one important question is: what sort of experiment, natural or contrived, might yield evidence? I’m thinking here of this sort of paper.
3. How big a problem is this? Again, we need numbers. For example, if imprisonment causes more offending than non-custodial sentencing, how much more? How big a problem is bad prison policy. Or, if Pakistanis are disproportionately likely to groom young girls, how many girls are involved? What are the chances of a girl falling victim, relative to suffering other fates?
Now, I don’t know how such a campaign would change politics. I suspect that, very often, it would reveal just how much we don’t know. But it would revolutionize the media. TV and radio would have to drop all those blowhards who just exchange anecdote and hypothesis; in proper science, no-one give a damn abot anyone's opinion - it's the evidence that matters. And it might even reverse the inequality of pay and status between columnists and reporters.

Comments

[Limeyankee]

This must happen. Please.

[The Silent Sceptic]

Feynmann on Cargo Cult science was pretty much bang on the money here.

http://calteches.library.caltech.edu/51/2/CargoCult.pdf

I think that the problem would be that many people don't understand how to assess evidence. I can't decide whether this includes politicians, or whether the politicians can assess evidence but choose to play to the gallery of those who can't.

Still, at least introducing the concept that there is a better way than just banging on about anecdata would be a step in the right direction.

[Liam Murray]

First off if you get 'less lazy' let me know and I'll sign up. Secondly I'm also a fan of Harford etc. but agree there's probably a gap here (although I love it I'm often shouting at the radio, they could rebrand it "It Depends" because they seem curiosuly scared of commitment).

But allow me a general, lazy point not supported by a shred of evidence. In my day job (wallowing in MI for senior managers in the financial services industry) I regularly see data abused & misused but - contrary to the common trope - that abuse isn't always statistically ignorant or erroneous abuse. It's often quite sophisticated, statistically sound even in terms of how it handles data. Nonetheless it's being used to advance an agenda (or halt another) because it's suits the person to do so.

I've never been able to articulate this well (clearly) but I have a nagging sense that it's possible for something to be significant and worthy of attention even when in statistical terms it wouldn't remotely clear those bars. Or, to put it another way, it doesn't follow that an agenda advanced in a statistically sound way is necessarily an admirable one.

[Mike Woodhouse]

If I was "in charge" I'd be most worried that such a study, or set of studies, would show that there's no "one size fits all" solution and that, say, some individuals might be most beneficially influenced by custodial sentencing, others by the non-custodial. Then where would we be?

[Edward Spalton]

If you look at the crime rate and growth of numbers in professions like criminology, probation and social work, there appears to be a correlation.

As numbers of "caring professionals" and "experts" increase, so does crime.

A little bit like CO2 and global warming really.
There strong evidence shows that oceans release carbon dioxide after things have got a bit warmer.

Which causes the growth in which in the crime/social work relationship?

[BenSix]

I'd lend a hand!

Still, I'm not sure (1) is obligatory to a claim. Let's say there's a significant rise in nocturnal mortalities in Springwood, Ohio. One wouldn't need to have located a plausible mechanism before getting faintly nervous as bedtime draws near.

[Tim]

Please keep me updated

[Ralph Musgrave]

Chris, More power to your elbow. On this topic and given your extensive knowledge of social sciences, can you enlighten us on exactly where the evidence is that looking at or storing child porn images leads to the real problem, namely actually taking part in sexual activities with under-age persons?

[charlieman]

Edward Spalton: "If you look at the crime rate and growth of numbers in professions like criminology, probation and social work, there appears to be a correlation."

Events happen. Unless somebody counts how often an event occurs, we don't have the information to determine whether it is a problem or an advantage or amusing human behaviour. Regarding group grooming, nobody has published a count so we don't have the information to draw conclusions.

After we have counted events, we can analyse and draw conclusions. Often we count things for a while and then determine that the data is inconsequential. Most management data falls into this category but managers persist in its collection.

Domestic electricity demand peaks during cold Decembers and Januarys in the UK. There is a correlation. However electricity demand does not determine the weather.

Government (a rational one) employs police officers and social workers because it has counted events and chosen to employ people to respond to them. Some of those people count events which were previously unrecorded. So there is a cycle, but it is event driven rather than employment driven.

[D503]

Re "What is the mechanism through which a Pakistani background might predispose a man towards sex crime?"

http://www.telegraph.co.uk/news/worldnews/asia/afghanistan/8257943/Paedophilia-culturally-accepted-in-south-Afghanistan.html

Pakistan and Afghanistan are neighbours geographically, socially and culturally.

[Funfoid]

The one thing you don't touch on is the centrality of developing & refining concepts in applied social science. "Pakistani" in social science *data* is an abstraction that could mean dozens of different things. Its v. important to be reflexive about this. Good social science moves backwards & forwards between concepts and data. I don't see much scope in your scheme for this, yet its seems to me that the refining of analytic terms is the other side of the coin to data.

[Anon]

"all those blowhards who just exchange anecdote and hypothesis". isn't this an anecdotal statement?

[mark]

Excellent idea. It might be worth asking the Adam Smith Institute if they'd be interested in supporting emotionally or even financially. That's if you think they're a good fit.

[Davidrnewman]

For 1, discovering the mechanisms starts with qualitative research. Unlike naturalists like David Attenborough, we can ask people what they think. So anecdotes can help us formulate hypotheses. We generalise quantitative data to populations, qualitative data to theories (see Allen Lee's MISQ paper on Generalising Generalisability).