"Consistent with"

In discussing Paul Romer's wonderful concept of mathiness*, Peter Dorman criticizes economists' habit of declaring a theory successful merely because it is "consistent with" the evidence. His point deserves emphasis.

If a man has no money, this is "consistent with" the theory that he has given it away. But if in fact he has been robbed, that theory is grievously wrong. Mere consistency with the facts is not sufficient.

This is a point which some defenders of inequality miss. Of course, you can devise theories which are "consistent with" inequality arising from reasonable differences in choices and marginal products. Such theories, though, beg the question: is that how inequality really emerged?** And the answer, to put it mildly, is: only partially. It also arose from luck, inefficient selection, rigged markets, rent-seeking and outright theft.

However, it's not just defenders of inequality who make the "consistent with" error.

Rational choice theorists have sometimes been too keen to explain apparently odd behaviour as "consistent with" rational maximizing: the notion of rational addiction might be the most egregious example of this.

Equally, though, behaviouralists might sometimes be too quick to see behaviour as "consistent with" people suffering from cognitive biases and too slow to see instead that it might be due to rational behaviour subject to particular information sets or incentives: I'm pretty sure I've been guilty of this.

Quite often, the facts are consistent with either theory. For example, the well-attested momentum anomaly - the tendency for assets that have risen in price recently to continue rising - is "consistent with" both a cognitive bias (under-reaction) and with rational behaviour; fund managers' desire to avoid benchmark risk.

My point here should be well-known. The Duhem-Quine thesis warns us that facts under-determine theory: they are "consistent with" multiple theories. This is perhaps especially true when those facts are snapshots. For example, a Gini coefficient - being a mere snapshot of inequality - tells us nothing about how the inequality emerged.

So, how can we guard against the "consistent with" error? One thing we need is history: this helps tell us how things actually happened. And - horrific as it might seem to some economists - we also need sociology: we need to know how people actually behave and not merely that their behaviour is "consistent with" some theory. Economics, then, cannot be a stand-alone discipline but part of the social sciences and humanities - a point which is lost in the discipline's mathiness.

* I soooo wish this idea was around in the 80s: it would have explained a lot of my frustrations when I was a student.    

** Scott Sumner is not guilty of this error, but I fear that other "libertarians" are less scrupulous.

Comments

[Ralph Musgrave]

If Northcote Parkinson didn’t set out a law based on the above point, he should have done. The law would run something like:

“Economists will always tend to reduce the rigour required in economics papers because that increases the total number of potential papers that can be written by economists, which in turn helps keep economists employed.”

[Matt Moore]

Awesome post. I particularly agree that Economic History should be given a far greater role in the teaching of Economics.

[Luis Enrique]

hence why economists disagree so much.

btw anyone interested in how some (imho) v thoughtful economists think about these questions try this "a model of modelling":

http://itzhakgilboa.weebly.com/uploads/8/3/6/3/8363317/gpss_model_of_modeling.pdf

includes concept of weak and strong compatibility with data

[mike phelps]

A couple of people who have done a lot of work in trying to go beyond "consistent with" are Deborah Mayo and Aris Spanos, both at Virginia Tech. Mayo has come up with an operational definition of a severe test which slays the Duhemian monster and Spanos has shown how econometrics can be done in a wsy that allowslearning ffom data instead of quantification of preconceived ideas. Well worth checking out Mayo's Error Statistics website.

[JohnW]

The 80s? Try 2001-04.

[rogerh]

Any fool can come up with a 'consistent with' explanation. What tests a theory is what it successfully predicts will happen if this or that condition is changed. Some investigators have the luxury of devising an experiment but economists seem stuck with observing what people do as the world evolves. Sure, there are experiments involving a handful of students trading in some way over a shortish period but IMHO these are social science. Which begs the question 'what truly is economics', does the subject have a part concerned with the maths of Laffer curves etc and another part concerned with what people tend to do under this or that condition and are there yet other parts. Which begs a further question, would it be academically inconvenient to admit economics is a bit of a mish-mash.

[Paul]

Paul Romer's mathiness seems more like Pythagoreanism to me. I would've preferred to call this http://arxiv.org/abs/1307.7006 sort of thing mathiness. Anyway, Mayo hasn't slain the Duhemian monster (nor revived the frequentist one: https://telescoper.wordpress.com/2012/08/19/kuhn-the-irrationalist/ ;-). And the lack of opportunity for experimental economics is a nuisance but I wouldn't worry about it too much: http://www.preposterousuniverse.com/blog/2013/07/03/what-is-science/