Unlearning Economics has a nice piece on the limits of sophisticated empirical techniques and the virtues of eyeball econometrics, reminiscent of Dave Giles’ advice “Always, but always, plot your data.” One extension I’d make to this is the importance of Big Facts. Sometimes, it’s more important that a theory explain a single important fact – or at least be consistent with it – than a range of small facts.
Take, for example, the efficient markets hypothesis. Researchers have found countless small facts that seem to refute this – well over 100 anomalies at the last count. All these, however, run into the Big Fact – that fund managers do not beat the market. This is not inevitable: they could in theory out-perform at the expense of retail or overseas investors. So why don’t they?
It might well be because the anomalies are not, in fact, that strong. They might be just non-replicable patterns in noisy data (pdf). Or it might be that once traders learn of them, they get bid away – something that seems more true in the US than Europe. Or it might be that high dealing costs mean the anomalies can’t in fact be exploited in the real world. And perhaps those anomalies that are robust are in fact a reward for taking risk: the defensive (pdf) anomaly, for example, exposes fund managers to benchmark risk, the danger of losing their jobs because they under-perform in a bull market.
The Big Fact, then, alerts us to the possibility that the EMH is mostly true in at least one sense – though of course the market can be both “micro efficient” and “macro inefficient”.
Here’s a second example. A Big Fact is that the unemployed are significantly less happy than those in work. This is inconsistent with ideas that unemployment is voluntary: people should be happy if they’re on holiday. It thus refutes labour market-clearing real business cycle theories.
A third Big Fact is that mainstream economic forecasters have consistently failed to predict recessions – something which pre-dates the 2008 crisis – and in fact are much worse recession predictors than the simple yield curve. This might tell us that recessions are inherently unpredictable. Or it might tell us there’s something wrong with orthodox macroeconomics (and not just DSGE models). Whatever your story, it must fit this Big Fact.
A fourth Big Fact is that a significant slowdown in productivity and GDP growth has followed an increase in inequality (in the sense of the share of income going to the 1%). This might tell us that increased inequality has caused slower growth – perhaps because it’s the wrong sort of inequality, Or maybe something independent of inequality caused growth to slow. Whatever it is, we have a Big Fact which defenders of inequality need to confront.
No doubt you can add other examples.
My point here should be a trivial one. Not all facts are equal. And given that no theory in the social sciences fits all facts, I’d rather they fitted the big ones than the little ones. One useful thing to do when reading any theory is to ask: what is the biggest fact that supports this claim, and what is the biggest that is inconsistent with it?
"It thus refutes labour market-clearing real business cycle theories."
I hesitate to say this, because I am sure some lunatics can be found in the halls of economics departments who did take it literally (I'm looking at you Lucas*), but what sort of lunatic would seriously propose that labour markets clear and recessions happen because people respond to negative productivity shocks by choosing to work less?
If that element of RBC is defensible, it is as an easy way of filling in that side of a model whose real interest lies elsewhere, and should be consumed in the full knowledge that it in no way describes how people lose their jobs in recessions.
* top of my list of bad economics is Lucas' calculations of the welfare cost of recessions by taking that model literally
Posted by: Luis Enrique | December 18, 2017 at 04:03 PM
p.s. I seem to be the only person who was not impressed with UE's piece. As you intimate, every econometrics student is told to graph their data (and make heavy use of Stata's tabulate commands to look at average etc. by sub-sets) and yes descriptive statistics are probably undervalued but his example where a table is more informative than a regression has what half a dozen countries two data dimensions? Plenty of papers start by laying out some descriptive statistics. RDD is there to try to solve problems around causal inference, not merely to show that something changes around a discontinuity. Just another instance of confirmation bias from UE.
Posted by: Luis Enrique | December 18, 2017 at 04:13 PM
Well, RDD is there to solve causal inference about a particular outcome...by looking at how it changes around a discontinuity. My point is that if there's no reason to doubt the story being told from descriptive statistics then you can use something like RDD as a robustness check in an appendix, but any precision it offers is spurious - highly local, resting on unchallenged assumptions, and full of statistical noise. Ultimately there are far more important questions that should be filling the pages of 'top' economics journals.
I'd also dispute your far-reaching statement about econometrics, as I've both taught and been taught it and the story you're telling is not familiar.
Posted by: UnlearningEcon | December 18, 2017 at 07:33 PM
The part about voluntary employment is simply incorrect. The notion of voluntary employment is that people choose to be unemployed because a favorable-enough employment opportunity is unavailable to them. That is, conditional on their options, they choose to be unemployed. There is nothing about them being "happy" to be unemployed.
Posted by: Howard Wall | December 18, 2017 at 10:02 PM
Perhaps then, Howard, it is more precise to say RBC assumes both the employed and unemployed face no cost or risk in changing jobs. There are no signalling effects, for example, of an employer offering a scandalously low wage or an employee being willing to accept it. RBC folk don' need no stinking asymmetric information.
But of course much of modern labour economics is about trying to quantify exactly these sort of effects, and studied ignorance of that is where Lucas' ridiculous estimates of the minimal welfare effects of a recession come from.
BTW "modern" New Keynesian models get their downward wage inflexibility feature from nowhere (they simply sprinkle some Calvo fairy dust on those RBC models) and so fail to incorporate the (often large) monopsony power of employers properly. That is why they still tend to find that a percentage point rise in inflation is more painful than a percentage point rise in the unemployment rate; something any man on the Clapham omnibus could tell them is reality-defying.
Now tell me again which class gains from maximising the measured welfare cost of inflation and minimising the welfare cost of unemployment?
Posted by: derrida derider | December 19, 2017 at 05:32 AM
UE that's gibberish. Finding evidence of causal effects is a perfectly sensible thing for top economics journals to concern themselves with, there are lot of things out there that people think they know but really lack evidence for.
You cannot infer the magnitude of an effect by inspecting descriptive statistics. Causal inference is warranted by the assumption that in the vicinity of the discontinuity "all else is equal" so of course the estimate is local, in that sense. That assumption is not unchallenged, it is exactly the sort of thing economists would challenge if they thought it was wrong. I don't know what "full of statistical noise" means, if data has 'noise' in that effects everything including whatever you learn by looking at descriptive statistics but extracting sensible estimates from noisy data is kind of what statistics is about. And finally "highly spurious". This is your MO isn't it. As good an estimate of, say, the causal impacts of the Head Start program for poor kids, as you are likely to get, and for you it's "highly spurious". Utter bollocks.
If you really didn't get taught to graph and tabulate your data when you were being taught econometrics then you had a shit teacher. And I pity your students.
Posted by: Luis Enrique | December 19, 2017 at 10:25 AM
"All these, however, run into the Big Fact – that fund managers do not beat the market. This is not inevitable: they could in theory out-perform at the expense of retail or overseas investors. So why don’t they?"
They do, and they have. In the UK at least. Over the last 10 years, the IA UK All Companies Sector Average is up 89.33% (after charges, to yesterday's price point). The FTSE AllShare is up 86.34%. (Both are total return, i.e. dividends reinvested)
So it's not quite as Big a Fact as it seems.
Posted by: SimonC | December 19, 2017 at 02:26 PM
«should be consumed in the full knowledge that it in no way describes how people lose their jobs in recessions.»
Oh please this is an endless and pointless discussion -- the RBC people and Lucas, in the terms in which they put it, make an entirely correct claim that essentially all unemployment is voluntary:
* if the so-called "involuntary" unemployed were willing to work for £1 per month I am pretty much sure that they would all find jobs;
* if they were willing to pay say £400 a month to their employer, I am pretty sure that the market would supply jobs to those willing to pay to work them.
The debate about "involuntary unemployment" is poisoned by the stupid intellectual dishonesty of those who argue as if the unemployed wanted work instead of wages, and the facile intellectual dishonesty of those who correctly point out that if the unemployed really wanted work instead of wages, they could always buy it.
Posted by: Blissex | December 19, 2017 at 06:59 PM
«The debate about "involuntary unemployment"»
As previously mentioned IIRC, I regularly have the same argument with the "Sandwichman", who keeps repeating that it is a fallacy to claim that the "lump of labour is a fallacy", because:
* the "lump of labour" is obviously a fallacy, in that the amount of labour demanded is pretty much without limit if pay is low enough or negative;
* the "lump of wages" is not a fallacy, it is much harder to increase the total amount of wages paid to workers than the total amount of hours worked, never mind the total number of "jobs".
Posted by: Blissex | December 19, 2017 at 07:24 PM
LOL OK Luis, I won't derail Chris' thread any more. I'll let people judge your post for themselves.
Posted by: UnlearningEcon | December 19, 2017 at 08:19 PM
A theory of the connection between inequality rising and productivity growth being poor is easy to invent. One could argue that inequality is a measure of monopoly power and political influence of the wealthy. More monopoly and high end income/ wealth equates to less competition and innovation. Also to politically driven crony capitalism where the ruling faction bribes their base with sweetheart arrangements which disadvantage other businesses and consumers/workers. The deliberate promotion of low pay makes demand overdependent on workers borrowing thus making the financial system overleveraged. The policy of making property into a speculative asset diverts capital away from employments which increase marketable output in favour of increasing the price of land so owners can make a capital gain.Using private firms to do state functions is featherbedding inefficiency.
Plenty of ideas to explore.
Posted by: Keith | December 19, 2017 at 08:55 PM
On and on I find myself puzzled by the EMH claim that "not beating the market" must be interpreted as informational efficiency. Isn't that an absurd idea? We can't beat the roulette either, but nobody claims for casino efficiency. Why that simple fact woun't account for anomalies AND and difficulties to beat the market?
Posted by: Pablo Mira | December 21, 2017 at 06:09 PM
«We can't beat the roulette either, but nobody claims for casino efficiency.»
That seems to me a good point, a better than most; after all there is also ample literature that stock prices in the short term are noise. After all there is a well-known book on the EMH with a title like "A Random Walk Down Wall Street".
The problem with your point and many others is that there is not just one EMH, but many, with definitions from rather weak to rather strong, depending on how cleverly it is worded, and while some of the weak EMH definitions are well supported by some evidence, this is often turned into claims that there is evidential support for the narrower definitions too.
Posted by: Blissex | December 22, 2017 at 11:35 AM