What do earnings tell us?

Sam Gyimah’s proposal that universities be ranked by graduates’ earnings has been criticized as missing the point that the purpose of a university education is more than personal enrichment. There are, however, other problems with it. I’ll take just seven:

First, at what stage of graduates’ careers do you measure earnings? If you do so soon after graduation, a course that produces lots of (say) investment bankers will seem better than one that produces lots of (say) barristers, because bankers earn big money quickly whilst barristers take longer to make it. If, however, you measure earnings later in life you risk using out-of-date information. The earnings of 50-somethings might tell us something about what university courses were like in the 1980s, but what does that tell us about them today?

Secondly, there’s a massive variation in earnings. Suppose that most graduates from a course earn average money, whilst one makes a fortune. Mean earnings will then be high, but median ones low. Which is the better guide for a student? The more we live in a “winner-take-all” or superstar economy, the bigger this problem will be.

Thirdly, qualifications alone do not determine earnings: a passing glance at your contemporaries will tell you this. As Bowles, Gintis and Osborne wrote in a classic paper (pdf):

Apparently similar individuals receive quite different earnings: a person’s age, years of schooling, years of labor market experience, parents’ level of schooling, occupation and income tell us surprisingly little about the individual’s earnings.

Many other things matter as well: soft skills; agreeableness (“being nice doesn’t pay” says (pdf) Guido Heineck); narcissism; psychopathy; looks; and so on.

Maybe differences in all these will cancel out over large numbers. If, however, we are looking at particular courses in particular universities we are not dealing with very large numbers. And if average earnings are skewed by a few high earners, such differences will distort the results.

Fourthly, earnings can be enhanced by irrationality. In particular, overconfident people are likely to do better than ones with a rational assessment of their ability. A good university – surely – would teach its students to be rational. To the extent that it succeeds, though, it might impair its students’ earnings relative to a university that inculcates unjustified confidence. To this extent, earnings-based rankings will favour bad universities over good.

Fifthly, and perhaps relatedly, we know that people from posh backgrounds earn more than those from poor ones even with the same qualifications. If we do not correct for this, a university that accepts more posh students will look like a better one than one that accepts fewer.

Sixthly, even if we grant that we could measure how universities enhance earnings, by controlling for all the other factors that influence earnings, what would this tell us? It would tell us what universities were like a few years ago. But what use is that to someone considering attending now? What if a great lecturer or two has left, or arrived?  

Seventhly, any university worth the name will encourage a sense of curiosity in its students. This, however, might not enhance their earnings. Lawyers might take on interesting pro bono work rather than stick to the dull day job; investment bankers will get bored and look for more interesting if less remunerative careers; or people might just ditch 60-hour weeks to do give themselves time for other things. Adam Smith wrote about compensating advantages because there really are such things.

Ranking universities by graduate earnings is, therefore, daft. It fails to heed the message of Jerry Muller’s new book, The Tyranny of Metrics. “What is most easily measured is rarely what is most important” he says. And, he adds, metrics can be gamed: all the above suggests that if universities are judged by earnings, they’ll want posher students; will encourage them to enter careers where financial success comes earlier; and will discourage rationality and curiosity.

 When people are stupid, though, we must ask why. I suspect Mr Gyimah’s proposal is founded upon an unwillingness to face two uncomfortable facts.

One is that teenagers’ decisions to invest in their “human capital” are in many cases taken in conditions of almost perfect ignorance – not only about the future shape of the economy but also about their future tastes and preferences. A desire to rank universities by some objective-sounding measure represents an attempt to create an illusion of knowledge that hides this overwhelming uncertainty.

Also, our earnings are only very partially determined by our own efforts and choices. Luck plays an important role and merit might sometimes even be a handicap. Naturally, though, Tories want to overlook this fact.

Comments

[TickyW]

In short, employers are being asked to judge teaching quality. How can they possibly know unless they observe the teaching as it takes place?

Keep business out of education!

[Dave Timoney]

If the idea really was to establish a link between individual courses and earnings, then you could have stopped after your first objection.

The inescapable delay (which it's hard to imagine could be less than 10 years) means that the price signal could only ever be a faint echo at best and one that would be a weak spur for improvement. After all, why would a lecturer bother to invest their effort if the results won't be seen for a decade and they may well end up moving elsewhere before then?

In fact, the mooted extension to the TEF appears to be another opaque "balanced scorecard", incorporating drop-out rates, nebulous "satisfaction" and equally nebulous "prospects" as well as earnings. It's pretty obvious that this will be less than rigorously empirical and that "judgement" will tip the scales (I suspect this is where the OfS may find a role).

The purpose of the exercise is clearly ranking, and the reduction of supposedly sophisticated metrics to a gold, silver or bronze label tells us that the political goal is a new tripartite division that will, to no one's surprise, mirror the old: the Russell group, the old redbricks and the polyversities.

As these course-level rankings will inevitably be aggregated, one likely consequence of this system would be to discourage gold-star institutions from investing in new, and thus risky, courses that might negatively affect the average in the short-term. This might be to the advantage of bronze-star universities, for whom the only way is up, but I suspect that they will struggle to break the glass ceiling. After all, if everyone gets a gold, there would be no point in the ranking.

[Blissex]

«employers are being asked to judge teaching quality»

Such naivety is always charming, but a more realistic interpretation is that employers are being invited to specify the ideological content of university teaching and endorse (or not...) the career prospects of university lecturers.

[Danny]

Yet more of the depressing mindset projecting University courses simply as training grounds for jobs, rather than about education and discovery.

[Brian]

Sam Gyima I remember from my time as an undergraduate. It is with total horror that I see this type of person in charge of such important decisions. I am much younger than Rees Mogg, but I remember the type and am much disheartened.

[plashing vole]

45% of my students are from minority ethnic communities: their employment prospects are lower than those of white graduates because we live in a racist society. 94% of my students are from working-class backgrounds and 95% of them live in an area of economic deprivation. Jobs are scarce for all of them; wages are low; many of them have family ties that mean they can't leave. Lots of them want to do jobs that are socially useful but not highly-paid. In short, there are major structural reasons for post-graduation economic inequality…and yet I'm going to be judged for them.

None of the people driving this are stupid. They must therefore be cynical.

[Steve]

There's also the question of whether young people going to university are actually investing in human capital in the specific sense of acquiring skills and knowledge that will enhance their productivity and fitness for specific occupations. I support the view that in most cases this simply isn't true and the real point of the decision and investment is to get a certificate that sends a signal to employers about the kind of person you are. Ranking is very difficult with that also, because the only major difference between one degree and another viewed from that perspective is the institution (so social status and networking effects become vital).

[Blissex]

«get a certificate that sends a signal to employers about the kind of person you are [ ... ] only major difference between one degree and another viewed from that perspective is the institution»

There are a few papers and books that argue persuasively that employers in most cases care almost only the degree of selectivity (whether by exam result or social exclusivity) of admissions. Put another way they see university admissions as a filter as they typically have dozens to hundreds of applicants for every role.

[Blissex]

«Jobs are scarce for all of them; wages are low [ ... ] None of the people driving this are stupid. They must therefore be cynical.»

Never underestimate the amount of effort and cunning sharp-elbowed middle/upper-middle class mothers can put into maximizing the value of their investments.

«and yet I'm going to be judged for them.»

More probably you are being judged for not having managed to secure a place at a more selective institution. The logic is that nobody willingly relegate themselves to teaching those that in reaganian or thatcherite jargon are known as "losers" or "future renters" ;-)

[Roger]

As someone who led innovation and change at a major institution, this list looks like the typical type of cynicism of change and "it can never work."

Your list is actually just a checklist of what would distinguish better measures from naive ones: make sure it tracks over time, include medians and averages, adjust it for other relevant data and categories, don’t use it as the only measure, and so on.

I suggest we would learn more by actually trying it in a mature and reasonable fashion and adjusting to experience as a partial measurement of something which is much, much better than what we have now.