Reinhart & Rogoff: true Keynesians

Any Keynesians who are rejoicing in the debunking of Ken Rogoff's and Carmen Reinhart's claim that high government debt reduces GDP growth should stop. I suspect Reinhart and Rogoff are, in fact, guilty of a Keynesian error.

To recap, they claimed (pdf) that ratios of government debt to GDP over more than 90% are associated with significantly lower average GDP growth. However, when Thomas Herndon, Michael Ash and Robert Pollin tried to replicate this result, they found that it was due largely to excluding Australian, Canadian and New Zealand experience in the late 40s, and an Excel error - which Reinhart and Rogoff admit was a "significant lapse"  - which led to good Belgian growth being excluded.Fixing these errors shows that high government debt has been compatible with decent growth.

So, why do I say Reinhart and Rogoff are Keynesians? Simple. The issue here is not really one of policy; insofar as Reinhart and Rogoff's result has been used by policy-makers, it is in the way that drunks use lamp-posts - as support rather than illumination.

Instead, the issue is about the culture of economics. And here, Keynes (among others) had an unfortunate if perhaps inadvertent effect. 

What leaps out of his General Theory is that it is entirely unencumbered by empirical evidence. Keynes thus helped to promote an ideal of the economist as a brilliant man capable of solving problems from his armchair by dint of superb intellect*. What got undervalued in this was the importance of the mundane grunt work of careful fact-gathering.

Reinhart and Rogoff's errors, I suspect, reflect a culture which prizes brilliance - and no-one doubts that Rogoff is brilliant - over dull pedantry.

When I started work, I realized that the job of the practical economist was not so much about theorizing but more concerned with gathering and understanding data - something which academe had wholly unprepared me for. For me, the difference between the professional economist and the amateur is that the former knows the numbers not in the sense of understanding high econometric theory, but in the sense of knowing where they are, what they mean and what they don't. This is no small skill. An ability to navigate the ONS website without recourse to language you wouldn't use in front of your mother requires a mastery of the arcane which is not given to many mortals.

I suspect things have changed a little since I was a student; a lot of good recent work in economics has involved using big data or generating facts by experiment - for example in the work of Mark Grinblatt, Mattias Sutter or Esther Duflo to name but three of many. But I'm not sure it's changing enough. Diane Coyle has reported that many employers want economics students to have "a better practical grasp of quantitative methods including collecting and understanding data (as opposed to more sophisticated econometric techniques)." Perhaps, then, the tendency to elevate the brilliant theorist over the empiricist lingers.

Traditionally, many economists have tried to model themselves on Isaac Newton - albeit emulating his autistic misanthropy better than his genius. But perhaps Darwin would be a better model. His greatness lay not so much in the theory of evolution - something quite trivial he took from economics - but in the years of careful fact-gathering that preceded it. Perhaps if this were so, the sort of mistake made by Reinhart and Rogoff wouldn't have happened.

* There's a contrast here between Keynes and Kalecki; the latter did develop hypotheses which he tested against data, which is is why I hold him in higher esteem than Keynes.

Comments

[vavoida]

the issue is about the culture of economics,
yes indeed, it's 2013 the data of economic papers is still not shared ...

at least the Excel error could have been avoided - see

Open Access to Data: An Ideal Professed but Not Practised

Out of the sample, 435 researchers (89.14%) neither have a data&code section nor indicate whether and where their data is available. We find that 8.81% of researchers share some of their data whereas only 2.05% fully share.

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2224146

[Luis Enrique]

I don't know. That R&R paper didn't contain any clever theorising, it was just "glorified descriptive statistics" as the saying goes. They say those excluded countries were "explained by the fact there were still gaps in our public data debt set at the time of this paper" (I'd have thought there was a case for deliberately excluding all post-WWII data, but that's a different argument). The weighting choices are debatable. But things like how to weight data are precisely the sort of things economists tend to think hard about.

When it comes to questions that involve data like public debt and GDP growth, we have to rely on the statistical agencies - you surely aren't complaining that economists don't go out and try to independently construct such data are you? And economists are quite involved in thinking about what statistical agencies do and how they do it.

what data is this oft repeated claim that the economist profession doesn't place enough weight on data, come from? Sometimes I think it's just prejudice. Of course I could cite very many papers that feature economists paying attention to data. If we were to survey all economics papers published, divide them into those that pay close attention to data and those that don't, what proportion would count as "placing enough weight on data"?

Gathering new data is harder, but producing an interesting new dataset (and doing something interesting with it) is a great way to get published in a top journal.

[Luis Enrique]

also, didn't R&R do some dull data gathering for their JEP paper dataset?

http://www.aeaweb.org/articles.php?doi=10.1257/jep.26.3.69

I think you might have hung your point on an unsuitable peg

[chris]

@ Luis - I'm not saying R&R were distracted by high theory in that paper. I'm saying they were socialized into a discipline/culture which undervalued data gathering and thus led even very good economists to be under-skilled in it. Pretty much all the founders of modern economics - Samuelson, Arrow, Debreu etc - were not data gatherers. That must affect the culture of people of Rogoff's age.
Yes, new interesting datasets are great ways of getting published. But that's a sign of the slow change in economics.

[Migeru]

"What leaps out of his General Theory is that it is entirely unencumbered by empirical evidence."

Excuse me, what?

A cursory look at the General Theory http://gutenberg.net.au/ebooks03/0300071h/printall.html reveals discussions of empirical investigations of national accounts, in particular gross and net investment (such as they were in the early 1930s) in Chapter 8, section IV.

The e-book I link to even has as an appendix a 1936 paper with addenda to the data on investment.

Econometrics was in its infancy back then, but that doesn't mean that Keynes wasn't aware of the literature or that he didn't quote it and refer to it in the General Theory.

[Keith]

Getting elementary sums wrong is just sloppy. It is unacceptable in any academic discipline. If it is more prevalent among economists who discuss public policy affecting millions of people then it does indeed show up something very wrong about economics. But most people not indoctrinated in "neo liberal" "neoclassical" what not have probably had a suspicion about this ever since the GFC blew up modern economic phantasy land. But the economics profession and state insist on driving down living standards on the cross of Gold sorry Capitalist economics.

You can go back to your DSGE model now and admire perfectly efficient markets solve all problems with an imaginary auctioneer to make the sums add up. Off course if you had an actual auctioneer it would avoid getting your spreadsheet in a muddle. May be the auctioneer needs maths lessons? Why did you not anticipate that based on your rational expectations?

[chris]

@ Migeru. Thanks for pointing that out. You're right that Keynes does discuss data on net & gross investment in ch 8. But I'm not sure that data bears much upon what is distinctive about Keynes' theory. He himself writes immediate after the discussion: "the above is, to some extent, a digression."
The appendices were written after the General Theory and originally appeared separately - and one (app 3) actually discusses empirical evidence against the GT's claim that real and money wages usually move in opposite directions.

[Luis Enrique]

Well I think you're certainly right that brilliant theorists are regarded as, well, brilliant, whilst dull and pedantic data work, whilst no doubt very useful, isn't regarded as particularly difficult, so generally isn't as impressive, in a sense (although some of the data people think to gather reveals a touch of genius). There's something inevitable about that, I think, and it's not just economics. It's a bit like complaining the star striker is more highly valued than the club groundsman.

[Andrew]

Luis, the paper linked in the first comment is damning and supports my impression coming from a science background, and the thrust of this post, that economics has a long way to go to become a respectable discipline.

Chris - how exactly did Darwin "take" the concept of evolution by natural selection from economics?

[boilerman]

Basic errors from economists do undermine their arguments but what I often find frustrating is their inability to explain the real world mechanisms for changes in their models.

I occasionally attend briefings from our local agent of the Bank of England on their quarterly inflation report. Over the last year or so we have seen a flatlining in GDP but an increase in employment. The maths of this result in a fall in overall national productivity.

This has been presented as a bit of a mystery to us non-economists and I am still none the wiser on the drivers for this.

It was asked whether this could be a rebalancing of the UK economy from less labour intensive sectors (eg financial services) to more labout intensive (manufacturing and services) but the answer was of the don't know variety.

[Wise Monkey]

@Andrew (and Chris for that matter), If you're from a science background then you'll know that extrapolating from one observation isn't sensible. Tempting as it is, this debacle does not establish lack of respectability, and says nothing about the culture of economics. Data lapses occur in the natural sciences too.

[Luis Enrique]

most everybody agrees that code and data availability needs to improve - actually the code bit is more important, because data is often compiled from public datasets, so can be replicated with a bit of effort even if the authors don't make the data they used available (and many will email it to you on request even if it's not on their website). But there are tons of papers out there that are essentially replications - or "robustness testing" - of previous results. It really is not the case that economics lacks a culture of replicating results. In my little subfield - foreign aid - the key results have been pored over ad infinitum.

Of course I don't have the data to answer this question, but my guess would be that the gulf with other sciences isn't so large. First, I'd guess that things like coding errors are present in published work in all disciplines. Second, data availability is one thing, actually doing the replication is another. Some people might be surprised to know that peer reviews do not tend to involve going through each line of code to look for mistakes - reviewers simply don't have time for that. Also I am under the impression that in many sciences, getting hold of people's data isn't enough, because what you need to replicate is the experiment that generated the data. How often does that happen? Ioannidis, one of the authors of these papers criticizing economists for poor replication - really made his name making the same point about medicine. Although maybe in some disciplines (physics?) you have researchers with similar equipment working on the same problems so replications happen all the time?

[chris]

@ Andrew - Darwin read Malthus's Essay on Population, in which he describes how people compete for a scarce food source, and elaborated on it:
http://www.ucmp.berkeley.edu/history/malthus.html

[Andrew]

@Chris - thanks I thought you must be referring to Malthus. I agree very much with your valuing of Darwin's (and many other Victorian scientists') monumental body of evidence gathering.

However, the simple observation of competition for resources by offspring is in no way comparable to the notion of speciation by natural selection of inherited characteristics.

Malthus included no element of variable inherited traits or selection of them, let alone the fairly mind-blowing insight that this imperceptible effect could work upon a single common ancestor to result in the diversity of life observed today. This is a very deep mechanism that must apply to all replicating systems. It seems simple once you are familiar with it like many deep insights.

All Malthus did with his observation of scarcity was turn it into a Divine stick, supposedly used to encourage Man to keep to the straight and narrow.

Every scientific insight relies on prior thought. Indeed others were thinking along evolutionary lines at the time, partly influenced by Malthus. But to suggest that Darwin simply "took" his theory from economics is a ridiculous overstatement, which I assume was partly tongue-in-cheek!

[Andrew]

@Luis - Ioannidis's criticisms of medical research are almost entirely focused on the misapplication of statistics - particularly the inappropriate implications drawn from "statistically significant" p-values when not assessed in the light of prior probability of hypotheses, existence of other studies, and bias. These are important points, actually quite difficult to deal with, and much effort is being made to address them. (e.g. registering of all studies to address reporting bias, standardised metaanalyses). Of course, most research is of little value, and I imagine only a tiny minority is attempted to be replicated, but the important positive findings generally are.

In economics Ioannidis is talking about not even sharing data and methods. Never mind the interpretation of results within a wider research context.

There is really no comparison.

Another way economics fails as a discipline is in the lack of widespread registration of conflicting interests.

Another way it fails is that neoclassical theory is internally incoherent, aka wrong. Although it seems this is being slowly addressed at the fringes (including by a textbook you linked to at one point), the progress seems utterly glacial by scientific standards.

[Andrew]

@wisemonkey - I was ignoring this particular error, not extrapolating from it. Everyone makes mistakes, but economics does not have an effective scientific culture. In its unflattering historical mainstream, it seems more like applied 19th century mathematics and philosophy for people who weren't very good at mathematics.

I agree there seem to be noble efforts being made at the fringes.

[Luis Enrique]

oh right - who is it that pointed out lots of medical research isn't replicated and often cannot be when attempted? I thought it was Ioannidis.

I don't think you know as much about the culture of economics as you imagine.

[Wise Monkey]

@Andrew - no disrespect, but really now you're resorting to empty/incorrect assertions.

What on earth is the 'Unflattering historical mainstream'?

What is your evidence for its 'reliance on applied 19th century mathematics'?

And even if it appears to yourself as 'philosophy for people who weren't very good at maths' (again what do you mean here? Do you have evidence?), can you explain why philosophy done by people who aren't very good at maths is inferior to those who are?

Pick up any recent mainstream economics journal and you will find it full of probability theory (1920s and 1930s at the earliest, and ongoing), Bellman equations (1957), and of course ongoing econometric theory representing the frontier of research in statistics.

Given that you don't have a clue what is going on in economics research, I find it hard to take seriously your surmise that nobility is only to be found in the fringes.

[Andrew]

@Luis - yes Ioannidis pointed that out, but I don't see how that conflicts with anything I said. I said most of it isn't replicated - it can't be, there is too much and too much of little value. However almost always the very important pieces are attempted to be replicated. Indeed many of Ioannidis's discussions point to exciting and much-trumpeted early results that quickly lead to disappointment. They couldn't do so without failed attempts at replication.

I don't know much about economics, it's true. I'm offering an outsiders perspective - that relative to many other disciplines it compares poorly. We can't be experts in everything, but if economics, unlike every other science, can't make some sort of useful sense to an intelligent outsider who has put a little effort into reading, then what precisely is its value?

The foundations of neoclassical microeconomics are simply drivel. Internally incoherent, and unbacked by empirical evidence.

Keynes made some valid criticisms of classial economics, but his major work is also in parts simply drivel, an exercise in obscurantism, partial analysis and undefined terms.

Sure, as far as I can see, there are many other streams of thought in modern economics, there is the Marxist lot, the Austrian lot, the interface with psychology, some people who actually try dynamic modelling of a dynamic system and large areas of actual empirical evidence gathering.

However that the traditional textbook foundation of the subject can be that bad is just amazing. And I find it absolutely incredible that the likes of Krugman can go around with his two-axis graphs with a couple of lines on them, expecting them to be a useful model for large sections of the world economy, and be regarded as some sort of intellectual prodigy! That the likes of him can essentially ignore credit creation in macroeconomic analysis!

Economists to this day don't share data and methods. Economists to this day don't register conflicts of interest.

It looks pretty bad from the outside. Now I'm all for someone correcting my impression if it is simply the result of ignorance, but at this stage I would find that hard to believe.

[Andrew]

@Wise monkey - Neoclassical economics has been an continues to be based on modelling equilibria using differential equations.

That is what is still in the textbooks.
That is what an outsider finds when looking for introductory material.

Pretty stupid.

Now no doubt you will tell me of the wonderful things happening at the cutting edge, but in medical textbooks there are now very few chapters devoted to the four humours.

[Andrew]

Here is a good example from Chris why economics is not very good. Chris picks his beautiful equation:

http://www.investorschronicle.co.uk/2012/04/17/comment/chris-dillow/investors-beautiful-equation-WugWtCOfBWT0OgTMwHKBxJ/article.html

That equation relies on standard deviations to measure variabilty in non-normally distributed variables, and a linear measure of risk preference and linear measure of correlation.

http://upload.wikimedia.org/wikipedia/commons/e/e0/Lorenz.png

Now *that's* a beautiful (set of three) equations.

[Wise Monkey]

@Andrew,

Your analogy of the four humours doesn't apply.

Your rejection of basic calculus is silly.

If you pick up any undergraduate textbook in pure maths or engineering, it too will be full of differential equations. Does this make mathematics and engineering 'Pretty Stupid'?

There's also a voluminous literature recognizing, and addressing, your concern of non-normally distributed variables. What's clever about that particular equation is the amount it can explain, and not that it explains everything. You need to understand this distinction.

There isn't much point in continuing this discussion. As you admit in your post addressed to Luis, you 'don't know much about economics, it's true'. But don't let your ignorance get in the way of a satisfying prejudice.

[Luke]

Going back to something I think Chris said about the age of RR, or at least the era in which they trained. Were they very unsophisticated in their use of excel and modern techniques generally? I have no idea (being v unsophisticated in that area) but I have seen comments like "they use excel for regressions? Wow, buy them a licence for xxxx."

I don't really know what that means or why it is so odd, but are younger, tecchier economists better at crunching data?