Bayes theorem: an application to Afghanistan

I said yesterday that MPs should know their Bayes’ theorem. To see its usefulness, let’s apply it to the question: do the deaths of British troops show that the mission in Afghanistan is failing?
Bayes’ theorem lets us answer this if we know just two numbers (well, actually four, but two are just one minus the others).
First, what did you think was the probability of success in Afghanistan before the mission began? This is the prior probability, which we’ll call Ps. The probability of failure, Pf, is one minus this.
Second, what is the probability that we’d see the number of deaths we have, if the mission were succeeding? Call this Pd|s. One minus this gives us Pd|f.
Let’s pin some numbers on this. Let’s call Ps 0.8. I guess this is the sort of number the government had in mind when the mission began. It can’t be much lower, because no-one would send troops to war without a high probability of success. But it can’t be much higher, because that would be over-confident, given that the British and Russian empires both failed to control Afghanistan.
Our second question is trickier. It could be that the mission was always going to be bloody, and in fact we’ve had fewer casualties than expected. On this view, Pd|s is high, and the deaths are no evidence of mission failure - in fact, the opposite. On the other hand, there was a vague hope that troops would “leave without firing a shot.” From this perspective, Pd|s is low, and the deaths are evidence of mission failure.
Let’s split the difference, and call Pd|s 0.5 so Pd|f is 0.5.
Now let’s take Bayes’ theorem to derive a number for the probability of the mission succeeding, in light of the casualities we‘ve suffered. This is Ps|d, the posterior probability:
            Ps|d = (Pd|s x Ps)/[(Pd|s x Ps) x (Pd|f x Pf)]
Plug the numbers into this, and we get:
            Ps|d = (0.5x0.8)/[(0.5x0.8) + (0.5 x 0.2)] = 0.8
In other words, the posterior probability is no different from the prior. The reason for this is simple. I’ve constructed the example so that the deaths are no evidence either way. So there’s no reason for the prior probability to change.
But what if the deaths were more consistent with failure? If Pd|s were, say, 0.2 and Pd|f 0.8, then our posterior probability would be 0.5.
Now, I write this not to take a view on Afghanistan. I know as much about Afghanistan as Abu Hamza does about playing the banjo. I do so rather to show how Bayes’ Theorem can help us think about the issue. I’d stress two uses of it.
First, it forces us to ask of any piece of information: what is this evidence of? Are deaths evidence of mission failure or not? It’s in this context that we need specific military expertise.
Second, we can reverse engineer the theorem. Rather than plug numbers into it and accept the answer, we can use it to ask: what numbers give us a high probability of success? How plausible are they? Again, military expertise can illuminate here.
Note that I am not saying that Bayes’ theorem is sufficient. It can’t define “success”. Nor can it tell us whether the success is worth the costs. It is, though, surely an important part of how to analyse the issue. And yet - in public at least - politicians do not think this way. They did not tell us Ps or Pd|s (or even ranges thereof) before the mission began. Nor do they tell us now. We can all think of reasons for this omission. But are they really satisfactory?

Comments

[Richard Mann]

I'm not sure its valid to say Pd|s = 1 - Pd|f. Probabilities should sum to 1 in the argument, not the condition. But other than that a very illuminating argument

[Chris Williams]

I'm not a probability theorist, but I can tell you that the British Empire casualties in WW1 reached a very high level in the last four months of 1918. And we know how that one turned out. The Battle of Berlin resulted in more than 200,000 soviet casualties too.

Whatever the merits of Bayesian analysis, this crude McNamarism as an index of anything much needs to be looked at very critically, if at all. I'd pick a better example next time, were I you, Chris D.

[David Heigham]

I half remember an ancient example (from the 1920s or 30s, I think)illiustrating the appliction of Bayes theorem to a roughly analagous situation. It ran something like this:
There are disturbances in a colony. The government sends in troops to pacify the area. The trend in the number of fatalities from violence is an indicator of whether the policy is succeeding or not. That is to say, in the absense of other indicators a decline in the number of fatalites from period to period indicates that the likelihood that the policy will succeed is rising from period to period. The other side of that coin is that a rising trend in fatalities indicates a declining likelihood of the policy succeeding.

The message I remeber from the example was that even if your prior (here for the likelihood of success) was not quantified, you could tell the direction in which it wes changing by looking at the evidence.

[Jackart]

D-day was the bloodiest day of the war as far as British military casualties were concerned. The more effort you make, the more casualties you take. In Afghanistan for example the Italians were thought to have stayed in base, buying peace with a taliban Danegeld. The French, taking over were not informed about the payments, and took casualtis as a result. The US and British forces (and latvians, Danes and others who are also fighting hard) are taking casualties, whilst fighting for the aim

War is a risky business, and I think, like Chris Williams, you should pick a better, more tasteful example.

[chris]

@ Chris, Jackart. Your examples do not invalidate the use of Bayes theorem. Quite the opposite.
The question: are high casualties a sign of success or failure? is surely important.
In the case of D-day, casualties were high because the operation was a high-stakes, high pay-off one. They were not, therefore, a sign of failure: in my example, Pd|s was high.
All Bayes theorem does is force us to ask this question.
I would have thought that when lives were at stake, it is more important to think properly.
It is precisely because war is a risky business that we need Bayesianism, as it is a huge part of how to think about risk.

[CharlieMcMenamin]

Chris,
you've got to help us non mathematical types who can't count passed 10 without removing our socks on this one.

I've read this post and the previous one with interest and I note that one link goes to a Wiki list of of cognitive biases which include 'Déformation professionnelle — the tendency to look at things according to the conventions of one's own profession, forgetting any broader point of view.'

So, forgive me, but do we have anything more here than an economist used to calculating risk probabilities telling us that politicians should routinely use a risk management technique?

You accept Bayes can't tell you what success is, just, it seems, measure it if you first agree that there is a cut off point of possible non optimal results/ consequences you're not prepared to accept.

Quite often, though, things in politics aren't like that at all. Some policies are not about results or consequences per se, but about keeping together invariable fissurous coalitions, each element of which will have a different point of Bayesian non optimal return on different issues. Scoring this sort of thing is very complex and, to be frank, hard to see the point of.

Isn't this exactly an example of 'Déformation professionnelle? Or am I just being very, very slow?

[jameshigham]

Note that I am not saying that Bayes’ theorem is sufficient.

Thank goodness for that.

[BG]

Chris,

Thanks for the interesting example. Not quite sure that the math works entirely though. As Richard points out, I don't really see why you would have Pd|s + Pd|f = 1. The rest seems fine, but that relationship doesn't have to hold I think (if d and s / f were independent for instance,that would imply that Pd = 0.5 no matter what, which doesn't really sound right).

[Keith]

Is this not an example of the misuse of Maths along the line criticised by J S Mill? Some things do not admit of numerical methods as they involve Judgement of a qualitative kind. To use numbers is an attempt to give a spurious certainty to uncertain prospects. Any adventure by Bush was obviously a bad idea. Since the Neocons are a bunch of Right wing lunies who think history is on their side; rather like other extremists. History however is not on any side. History is an ex post construct.

[chris]

@ Charlie - I don't think this is an example of deformation professionnelle. Bayes theorem is a way of thinking, used by many science professionals. In my example it explicitly invites us to draw upon another area of expertise, as only military expertise can tell us Ps and Ps|d.
You're right that politics is also about building coalitions as well. But surely knowing the (changing) probability of the success of a project matters.
@ Keith - this would be misusing maths, if it were used to generate a spurious certainty (or even a certainty about probabilities). But it's not. We can input ranges into the theorem, or use the theorem backwards ("You think the Afghan mission will be a success. What values of Ps and Ps|d generate this view, with what degree of certainty?)Or we could use it just as a reminder to update our views in light of evidence.