Equalizing exam grades: a defence

In the comments on this post, D2 objects to my proposal for equalizing exam results across schools. Employers, he says, would look at what school applicants attended. The result would be that an employer would overlook someone with good grades from Knowsley Comp in favour of someone with worse grades (but perhaps better actual scores) from Eton - in the same way that employers now prefer graduates from Cambridge over South Bank University. The bright kid from Knowsley would therefore lose the valuable signal of his brightness.
I'm not sure how strong this objection is, for three reasons:
1. Put yourself in the position of an employer choosing between a candidate with As from Knowsley Comp and one with Bs from Eton. You know - insofar as exam grades measure anything - that the Knowsley kid has unusual effort and ability for his type, and has the strength of character to resist peer pressure. He might also be exceptionally bright - good enough to get As anywhere. You also know that the Etonian is a second-rater of his type.
Sure, the latter might - only might - have higher absolute test scores. But many - not all, of course - employers would surely prefer the strength of character or natural intelligence of the Knowsley graduate. Under my scheme, this gets signalled better.
2. The analogy with universities is a weak one. Employers prefer Cambridge graduates to SBU graduates partly because being a Cambridge graduate signals that one had higher ability at age 18 than being an SBU graduate does - though the signal's imperfect. But the signal sent by the fact that you went to a "good" school rather than a "bad" one is that you have richer or pushier parents. Fewer employers would value this.
3. Equalizing grades across schools would help reduce differences between schools. Under the current system, parents have every incentive to seek out good schools. Under my system, this incentive is greatly blunted. Indeed, I suspect many parents would actually want their kids to go to bad schools, because they'd think Tarquin would have a better chance of getting As there. These parents would then put pressure on the school to raise average performance (insofar as this helps Tarquin as much as the other pupils.)
I suspect, then, that D2's objections aren't crushing. Sure, some employers would prefer the dull Etonian to the bright comprehensive lad - but perhaps no more than do so today.

Comments

[Shuggy]

"Indeed, I suspect many parents would actually want their kids to go to bad schools, because they'd think Tarquin would have a better chance of getting As there"

I doubt this. They'd be much more likely to worry about Tarquin getting his head kicked in and are unlikely to think this is a price worth paying for having a supposedly easier academic regime.

[dearieme]

"You know - insofar as exam grades measure anything"; my dear fellow, where have you been living? Exam grades measure in part the quality of the coursework you hand in, which depends largely on who actually wrote the work and how often the teacher then rewrote it for the pupil. One of my local state schools has a fine reputation for teachers teaching the pupils how to cheat in practical exams and an equally fine reputation for widespread cheating on coursework. One of our local private schools is alleged to be "straight" in these matters. How true? Possibly a decent approximation. How generalisable? Search me. But to assume that exams are like the exams you took in your youth - oh dear, you don't have children, do you?

[dsquared]

[You know - insofar as exam grades measure anything - that the Knowsley kid has unusual effort and ability for his type, and has the strength of character to resist peer pressure]

You don't know this though. You might know this if we also introduced a "superduper A***++" grade and reserved it for the top one or two individuals. As it is, the A grade would be given to the top 10%, and in an overall-low achieving school (I picked Knowsley because it's apparently one of the worst LEAs in the UK) this is going to be massively driven by the noise in the test results. It certainly is possible to have an entire school year in which there are *no* bright or diligent individuals and even the A****+++ performer in fact got the grade because of a reasonable attendance record.

[You also know that the Etonian is a second-rater of his type]

If we divide through by the class warrior stuff here, you also know that this type is an unusually motivated and high-achieving type. Total Network Solutions came first in the Principality Building Society League of Wales and Watford came last in the Premier League, but I would guess that the betting would be heavy on Watford in a match between the two.

[Employers prefer Cambridge graduates to SBU graduates partly because being a Cambridge graduate signals that one had higher ability at age 18 than being an SBU graduate does ]

Partly, perhaps, but they mostly prefer Cambridge graduates to SBU graduates because the Cambridge degree is harder, it is a more competitive environment and therefore it is sensible to assume that with equivalent grades, the Cambridge graduate has actually learned more. So I don't think that the analogy is all that weak. Bayesian table once more; set one up with sensible proportions of clever kids and thick kids in good schools versus bad schools and you'll see that it makes sense for an employer to adopt the rule of thumb that a bad school graduate is thick.

[Sure, some employers would prefer the dull Etonian to the bright comprehensive lad ]

But all employers would prefer the bright comprehensive lad to the dull lad from a really bad school, and under your system they'd get the same grades.

[dearieme]

Oh, and another thing. For the two schools I have in mind, many of the state school pupils tend to get private tutoring in the evening - you know, like Blair's children; the private school pupils, by reputation, get much less.

[Stuart]

Chris, you refer to Texan schools doing this. Is there any data as to whether this has increased or decreased absolute test scores? How long have the Texan schools been doing this for?

[Roger Thornhill]

Chris, I am very much in favour of equalisation as long as it means "65% correct = grade C", not "this school is crap so lets lower the percentage needed for a C" nonsense. The currency should be maintained for it to have value.

Of course employers will look at which school the kid attended anyhow. That is a given.

A kid with 10 B's from a bog comp vs 10 B's from Eaton? Who is likely to have more rigour and go-get? Bog Comp. Who is likely to have better contacts in the media? Eaton.

The issue is about transparency. Those 10 B's might also benefit with the intra-school positioning and also the precise score. So "top 5 of the entire school" would be very useful for employers - it would make me want to interview, for sure.

In Texas I suspect that the schools do not select, no?, so saying the top 10% get automatic access may be a way to mitigate against hopless schools that otherwise bright kids have to endure. It is an escape pod, as it were but not the same as grade equalisation, IMHO.

[Matthew]

Wouldn't it be far easier (though I still see enormous practical problems with people gaming the system) to keep the grading system as it is, but also append to the A-level certificate or whatever it is you get the percentile that the student came in the school's grade distribution (and I would limit it to 3 A-levels). So 1%, or 85%?

[Alastair]

As a parent, I would definitely agree with Shuggy

"They'd be much more likely to worry about Tarquin getting his head kicked in and are unlikely to think this is a price worth paying for having a supposedly easier academic regime."

[Bruce]

A nasty downside with is is that students would be competing with each other for the higher rankings. Not with students generally but with their classmates. Not likely to foster a postive learning environment.

[dsquared]

I hadn't thought of that; there is an implicit academic subsidy to bullying here. If I am the class bully, then every nerd and keener who I can bully out of the state education system increases my own chances of getting to university.

[Chris Williams]

"Tarquin getting his head kicked in and are unlikely to think this is a price worth paying for having a supposedly easier academic regime."

I thought that sending your sons off at age 7 to get:
(a) a decent education and
(b) their heads kicked in
was what the British upper-middle classes have been doing for about 150 years?

[Matthew]

Imagine the riots you would get if in late May a busload of hyper-intelligent E.European children turned up to take their GSCEs.

[pseudonymous]

Why not give two test results: one giving the "absolute" grade, and one ranking the student within his school?

Admittedly, you would still have the bullying subsidy.

[dsquared]

I also can't think of a sensible solution for kids who move schools.

[Matthew]

You'd have to weight the exams results of the schools he's been in (at the time) by the length of time in each.

[Sam]

"Under the current system, parents have every incentive to seek out good schools. Under my system, this incentive is greatly blunted."

Only insofar as you view school as a mechanism for aquiing exam results, and exam results as a mere signalling device to advertise your brilliance. If you're interested in some actual education, rather than just some grades, there's still a big incentive to choose a good school with similarly-minded pupils.

[dsquared]

[You'd have to weight the exams results of the schools he's been in (at the time) by the length of time in each. ]

I'm not 100% sure of this (and will need a considerable burst of energy in order to do the maths), but I suspect that this will lead to paradoxes of one sort or another, whereby it becomes impossible to sort pupils into deciles.

[Roger Thornhill]

Bruce:A nasty downside with is is that students would be competing with each other for the higher rankings.

Nasty? No, preparation for the real world.

[chris]

D2 - You're right that in a bad year, dullards from bad schools could get great grades. But how likely is this? And what's wrong with erring on the side of generosity to these people, when today we err on the side of meanness.
You're also right that a dullard in a terrible school would score as well as a bright lad from a merely poor school. This is an injustice to the latter. But isn't it a smaller injustice than what currently happens, where the latter scores worse than many thousands who went to better schools but are less bright and hardworking than him.
And if you're right that Etonians are high achievers, what's wrong with denying them exam grades - they're only a superfluous signal.
Dearieme - one beauty of equalizing grades across schools is that it'll reduce the incentive for teachers to cheat in exams, as the obsession with league tables will be much reduced.
Shuggy - you're right. It's only at the margin that parents will prefer the "bad" school to the "good" one. And the margin might not be very extensive.
What you've all done is show some imperfections in my proposal. The question is: are these greater than the imperfections in the current system? I'm not sure.

[dsquared]

[You're right that in a bad year, dullards from bad schools could get great grades. But how likely is this?]

Very likely indeed. I seem to remember being the only kid in a couple of my classes who got an A, and my school wasn't even a bad school.

[But isn't it a smaller injustice than what currently happens, where the latter scores worse than many thousands who went to better schools but are less bright and hardworking than him.]

Don't see this at all. Either way he gets screwed, and it's surely pure class war to say that it's worse that he gets screwed by people at worse schools than him than people at better schools than him. In many ways it's worse, because he doesn't even have a chance under your scheme; in the current scheme there is an incentive for the number one kid in a bad school to keep on trying in order to reach the highest standards, but in yours, there's nothing he can do.

[And if you're right that Etonians are high achievers, what's wrong with denying them exam grades - they're only a superfluous signal]

It's wrong precisely because the Etonians have already got a signal, and you're destroying the information value of the only other signal it's possible to get. The Etonians will be fine under your system; everyone will assume that grades down to D at Eton are fine. But conversely, everyone will assume that anything lower than an A from Ysgol David Hughes is worthless, and that even the As at Knowsley Park are a bit dodgy.

C'mon, Bayesian box. Say that at the end of the year, 20 out of 30 Etonians can do fractions. Say at the end of the year, only one kid in 30 at Knowsley Park can do fractions. Grade A is given to the top 10% of each year. You're an employer who thinks it's really important that the guy you hire has to be able to do fractions. There's a guy from Eton with a B and a guy from Knowsley Park with an A applied. What do you do?

[dsquared]

and regarding choice of schools, what's to stop the parents moving Tarquin into a really bad (or merely mediocre) school for the last term of his final year, to get the best of both worlds? (any scheme aimed at stopping this is going to end up causing a lot of other problems too).

[Roger Thornhill]

dsquared: this occurs now in the form of parents moving kids into a Bog Comp to increase their chances for Oxbridge.

More tinkering = more fiddling.