Algorithms & reification

The A levels U-turn has prompted the question: what went wrong with the algorithm?

The answer is: nothing, zip, diddly, jack.

What went wrong wasn’t the equation, but people. People create algorithms and they do so according to the principle garbage in, garbage out. The error here was not of high-level maths. It was a basic failure to appreciate the nature of statistics. Statistics cannot discover what isn’t there. And the information we really needed – how to compare students across schools – just wasn’t there. As the great Dan Davies puts it:

No statistical method in the world is going to be able to give you good results if the information you’re looking for is fundamentally not there in the dataset that you’re trying to extract it from

The problem wasn’t the algo. It was that the people in charge of it combined stupidity with class hatred. And it is of course not good enough for Williamson to claim that “Ofqual didn't deliver the system that we had been reassured and believed that would be in place.” Ofqual isn’t some organic food company delivering a strange-looking vegetable. It is, or should be, under government control.

What’s going on here is in fact an old and widespread error. Its what Georg Lukacs called reification – the process whereby “a relation between people takes on the character of a thing”. What actually happened was that people in power allocated A level results. That’s a relation between people. But to some, this relation appears as a thing, an algorithm. Which effaces the basic fact that those in power are the enemy of young working class people.

With algorithms and big data becoming increasingly important, the danger is that the reification fallacy will increasingly serve as a means of effacing the reality of class power.

In fact, of course, it has done so for decades. When Luddites smashed machines, they were transferring their anger from people to things – just as people do today when they rail against IT systems failures rather than against the mismanagement that create them.

And in 1845, Marx described how capitalism appears to people not as the product of human action but rather as “an alien power existing outside them, of the origin and goal of which they are ignorant, which they are no longer able to control.” Of course, this does not mean that specific individuals are always to blame as they are in the A level case: often, social outcomes are emergent, the product of human action but not human design. Nevertheless, they are human-made. 

A classic example of this reification is how some people use the phrase “market forces” to defend income inequality. What this misses is that “the market” is not some entity existing above us but is really only a relation between people. And it is a relation of power. As Rick said, “all pay is, ultimately, a function of power.” Bosses and bankers have power; they must be bribed not to misappropriate their firms’ assets. But care-workers do not – often because they are women and migrants who lack outside options.

All this, of course matters for the reason Marx thought it did. If we regard inequalities and inefficiencies as arising from abstractions – be they algos or market forces – we are apt to either attack the wrong target or, worse still, to resign ourselves to fate. We need not do either. Technology - which includes social technologies such as markets - is a human construct.

Comments

[Dave Timoney]

Ironically, given all the guff talked more generally about inscrutable AI, the problem with this particular algorithm is that it was utterly transparent, making the reality of the social relations clear to even a child.

[droog]

Spare a thought for the people who defended government and the algorith and got the rug pulled out from under them. They are the true victims.

[ltr]

This set of essays are exceptional; brilliant.

[ltr]

"is" exceptional...

[Dipper]

Yet again we find that a complex evolved ecosystem, in this case education and exams, cannot be ripped away and replaced with something as effective in an instant.

The problem is that ranking and grading children by the state is an inherently unjust, inaccurate, cruel and limiting thing to do. The beauty of exams is that they take away the responsibility of the state for this injustice and dump it on the child. Had a bad day? Your teacher didn't teach you properly? Not our problem. The exam doesn't lie.

Removing exams makes the grading and ranking a process owned unequivoicably by the state. For many Tories, this is not what the state should be doing. It should be enabling and investing in our children not ranking them.

So the Johnson problem is that the virus and the nation's response has required him to implement a heavy-state socialism, and of course it doesn't work. Notably no-one has been able to propose a better solution, and the process of giving into all those who say that really despite the evidence they were top students has undermined the system for years to come.

Inevitably those complaining loudest in the media and elsewhere are also those who demanded we lock down making exams impossible.

Lots of critics seem to think the government should have had a plan in its back pocket to rip up institutions, freedoms and liberties, and replace them with a bureaucratically intensive administrate system which simultaneously preserves freedoms and standards. The correct conclusion from this episode is not that a different government would have been able to do this task better, but that the task is inherently impossible.

[droog]

"So the Johnson problem is that the virus and the nation's response has required him to implement a heavy-state socialism, and of course it doesn't work"

The leftist-socialist platform starts with big investments in education, more teachers, more school meals, not sabotaging teacher's pensions, etc. All of that comes first before any mention of abolishing exams.

The Tory solution was to undercut the up-front investment for ten years and then come up with an unpersonal way to muffle the individual achievement of students who grew up in that environment.

This is not proof that socialism fails. It's a Tory worst-of-both-worlds scenarion: Cut funding and preserve the status quo of their relished institutions.

[Dipper]

@ droog

no. English education is doing very well on international standards. And this discussion is about the distribution of grades not the absolute level.

Simply saying that the solution is to magic up money and spend it on lots of things is not a solution too this particular challenge, or any challenge. It is not a political philosophy, it is the absence of a political philosophy.

[Paulc156]

No. English education is doing rather poorly on international standards.
Although it is true that it isn't all about money, some of it surely is. Slashing Sure Start for early years is a prime example of Tories utter disregard for early years education which set them back hugely. And countries that achieve the highest standards employ the best graduate teachers. Singapore for example (Finland has similar standard in this regard) employs those with the top 5% of grades. Presumably this 'does' cost $$.
Overly competitive exam reliant methods of teaching achieve two unenviable outcomes.
1.The least content and most stressed children, a league in which are near top!
2.Poor rankings at main school leaving age(15-16),
"In reading, the UK is 14th, up from 22nd in the previous tests three years ago
In science, the UK is 14th, up from 15th
In maths, the UK is 18th up from 27th...no significant change for reading or science, with scores remaining broadly similar in Pisa tests since 2006, despite fluctuations in rankings." PISA.

Andreas Schleicher, the OECD's education director, said, "more than many other countries, UK schools had a strong culture of individual competition rather than co-operation"
https://www.google.com/amp/s/www.bbc.co.uk/news/amp/education-50563833

There was notable improvement in maths over last 3 years...but if you look at the top performers in China and Singapore where of course the state exerts a heavy hand they are streets ahead. More worrying is that kids in England are amongst the most unhappy and stressed out.
That's what an early competitive exam obsessed school culture should logically deliver.

If we look at Europe then two of the countries that we should aspire to are Finland and the new star on the block, Estonia. Both of whom frown upon separating out kids of different abilities. Neither start school proper until age of 7. Both of whom test a lot less. Both education systems (like their Asian counterparts) have strict standards imposed from the state level.
They have more content kids achieving much higher standards.
Why Estonia does so well:
https://www.bbc.co.uk/news/education-50590581

[Hugh Mann]

Wasn't the system agreed by the teaching unions?

Mary Bousted - "these grade judgements would need to be made using a range of evidence, and we are pleased that the regulator has agreed. This means that grades won’t be based on mock exam results, or any other single piece of evidence alone. It is important now that the guidance on what teachers can use and how they come to these judgements is clear and consistent. The NEU will be working with Ofqual and awarding organisations to help make that happen"

https://www.fenews.co.uk/fevoices/44890-direction-issued-to-ofqual-about-calculating-summer-2020-grades-in-lieu-of-exam-results

https://neu.org.uk/press-releases/summer-2020-grades-gcse-and-level

And didn't Labour want to scrap teacher grade assessment altogether for uni offers?

https://www.bbc.co.uk/news/education-49330720

Incidentally I see that some teachers, unsurprisingly, would rather over predict grades for black students than be seeming to racially stereotype. Not a huge difference ("only" 34% i.e. (53.8*100)/40), but what's amazing is that the LOWEST over-prediction (for White students, natch) was still a massive 40%!

The teachers should all get great reviews, 100% grade prediction accuracy!

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/32412/11-1043-investigating-accuracy-predicted-a-level-grades.pdf

"White applicants had the highest grade prediction accuracy (53.0%) and the lowest over-prediction rate (40.4%) Black applicants had the lowest percentage accuracy with only 39.1% of grades accurately predicted. This group also had the highest over- and under-prediction rates (53.8% and 7.1% respectively)."

[Blissex]

Dear guys, I'll quote two great sayings:
* "stuff happens"
* "everything is is easy if someone else has to do it".

So I have some understanding that the present government is dealing with a difficult situation and mistakes are going to happen.

But despite this about the details, the big picture is the real problem:

* The core tory constituencies of the upper-middle and upper classes anyhow largely use independent schools, and are not too bothered about what happens to the "losers" in the state school system and their small chances of getting one of those few "good jobs".

* The exam crisis has gained so much relevance because the "good jobs" are so few; sharp-elbowed middle class mothers are desperate to maximize their investment in their children and know that exam results are so critical to having a small chance at getting one of those few "good jobs". When jobs with decent wages, good security, low stress were more plentiful a lot fewer people bothered to get degrees, usually only those who wanted to do professions like doctor or lawyer or researcher.

* In general while a university education is good for personal development, not many jobs require it. Mass university education has mostly happened because it reduces the official unemployment figures and diverts discussion from the scarcity of "good jobs" to the fight to get one of those few via access to one of the "top universities", thus spreading right-wing ideology.

[Hugh Mann]

" Mass university education has mostly happened because it reduces the official unemployment figures and diverts discussion from the scarcity of "good jobs" "

I beg to differ, it's been encouraged because it's another three years of 'left' * indoctrination - and the beauty is, they're paying for it themselves!

You have plenty of grads who may not know any British history, other than that it was bad (except the fighting Hitler, or were we fighting Bush?). I knew a few, 2-1s from Russell Group unis, on £14k in the call centre.

The expansion of unis was yet another case of Thatcher winning her economic wars and being utterly defeated in her culture wars. Like so many things that Blair took a l'outrance, it started under Thatcher.

Nowadays its sold to gullible sixth formers by their own schools and evil but personable people from the unis, who talk about a "graduate premium" which applied in the days when 5-7% went, but has now almost vanished.

* note that this 'left' indoctrination never mentions things like "standard of living" or 'real wages', hence the call centre guys.

[Blissex]

«I beg to differ, it's been encouraged because it's another three years of 'left' * indoctrination»

So the thatcherite governments of the past 40 years have had no role whatsoever in it? Was is a secret cabal of "trots" who made it happen?

I had the impression that mass university graduate and postgraduate education was invented by John Major (a secret trot?) to obfuscate the high youth unemployment during the 1990s crash, and was continued by Tony Blair (another secret trot?).
And it was continued by David Cameron (trots are everywhere apparently).

After 40 years if thatcherite governments in solid power it is utterly laughable to claim that the current state of the UK is in any respect the result of leftist policies, unless one believes that the trots or their masters in the ECJ of the EUSSR have developed remote mind control devices,

[Dipper]

@ Blissex 'After 40 years if thatcherite governments in solid power it is utterly laughable to claim that the current state of the UK is in any respect the result of leftist policies'

well ... we haven't had Thatcherite government in power since 1987. Thatcher abandoned it after that election, Major wasn't, and Blair definitely wasn't.

Famously, the right won the economic battle and the left won the social policy battle.

The cultural establishment is clearly Blairite in that they think that a lot of intelligent people can and should plan every aspect of UK life to produce a properly ordered 'fair' society. The current manifestation of this is Covid and the strange belief that somehow perfect solutions were available despite the massive uncertainty surrounding the disease.

Re the A Level results, there seems a complete absence of any proposed solution that is better than the one the government originally came up with.

[Dipper]

@ Paulc156

Firstly the standard of English education is not what is at issue here; rather it is the specific handling of A level results in the absence of exams.

Secondly, international comparisons in maths are suspect in that different nations take different approaches. Many have a very rigorous approach, whereas the UK has a more rounded approach which includes applications and uses. So an international comparison based round a rigorous approach will inevitably reflect badly on the UK.

And Estonia is tiny. Any international comparisons which praises tiny nations , or nations that have specific roles like Switzerland or Singapore, are worthless.

[Hugh Mann]

Blissex - read it slowly.

"The expansion of unis was yet another case of Thatcher winning her economic wars and being utterly defeated in her culture wars."

It would have been much better for British working people had she lost her economic wars and won her culture wars, but then many of her advisers and cabinet members were people with little knowledge of, and even less sympathy for, working class Brits.

[Paulc156]

@Supper
"And Estonia is tiny. Any international comparisons which praises tiny nations , or nations that have specific roles like Switzerland or Singapore, are worthless."

Ha ha. Of course of course. Don't worry PISA is hardly recognised and being so low in the comparisons is quite easy to deflect. You can refer to the small size of nations for instance when the results of the comparison looks awkward. Finland only has over a million kids in education so obviously shouldn't count either. Or nations like China which are clearly far too big! No problem, just go back to your former assertions based on ...whatever.
As for polling of levels of dissatisfaction and stress of schoolkids...just irrelevant. You can go back to expressing opinions which reflect well on your pteferred worldview. PISA probably just got confused. Lol. ;)

[Paulc156]

@Dipper
Apologies. Predictive text insists on calling you Supper.

[AllanW]

Back to the op-ed;

'What went wrong with the algorithm?'

Nothing. It did exactly what it was meant to do; increase inequality. The problem is that those instructing it did not expect this much push-back to their blatant manipulation.

[Hugh Mann]

"It did exactly what it was meant to do; increase inequality."

Any, like, evidence to support that assertion?

I don't think, after the Blair years, you can have the brass neck to accuse Tories of increasing inequality. Real male median wages are still lower now than when Blair took office. It's as if wages had stayed static between 1945 and 1968.

[AllanW]

Go away, Hugh Mann, we can see that you already 'know' all you need to know about any issue. The only unknown is who is paying you to write the crap you spout.

[Hugh Mann]

Allan W - At least I provide evidence to support my argument, which is more than you seem capable of.

Argument by assertion isn't argument at all.

[Shaftoe]

I believe that the difference between emergence and construction that appears in this text is nowhere near as easy to work with as the author seems to have in mind. Not "human designed" but "human-made" sounds like an unnecessary difference in formulation. But it makes a difference whether something was made by humans or whether it is made of humans. Emergence is very difficult to plan and predict. It is unclear whether change introduced by elements will result in something better (for whom?). So I probably also don't share the view that it is better to hit a wrong target than to surrender to fate ("worse still").