Using models

Gene Callahan makes a good point. Economic models, he says, are things you should use, not believe. I'd like to amplify this.

For me, economics is primarily a practical discipline. It is not about pompous armchair handwaving, but a practical guide to better decision-making in the the real world: policy-making and institutional design are subsets of this. However, the real world is a complex place. And the solution to complexity is often to satisfice - to pick decision-rules, or models, that are good enough. As someone once said, what matters is what works.

Take, for example, the efficient market hypothesis. Strictly speaking, this is not true: for me, the strongest evidence against it isn't windy theorizing about the Grossman-Stiglitz paradox but the empirical fact that momentum stocks beat the market. But it is useful. Sure, the investor who acts as if it were true would miss out on momentum profits (and on the out-performance of defensive stocks), but he'd avoid countless big mistakes such as paying high fees to mediocre fund managers; trading too much; overpaying for new flotations or "growth" stocks; and so on.

The EMH might not be true, but it's good enough for practical purposes*.

In fact, in a complex world, there is a positive danger in seeking the truth. Gene gives the example of LTCM whose trading rules were true until they suddenly weren't. There are other examples. Goldman Sach's David Viniar's famous bleat that "We were seeing things that were 25-standard deviation moves, several days in a row" expressed the fact that his risk management models were true for a particular dataset but false when that dataset changed.

This highlights an important and overlooked trade-off - between optimization and resilience. A model that gives us a true answer in some states of the world can give us fatally wrong ones when those states change: this is what happened to banks' risk management models in 2007-08. A model that was good enough - such as "shit happens" - might have served banks better** than ones that were "true" in the sense of perfectly fitting a small data sample.

My point here should be a trivial one. We should ask two questions of any model. One is: is it good enough? The other is: what's the worst that can happen if we trust it? As some guy once said, it's better to be roughly right than precisely wrong.

* especially if accompanied by the "sell in May, buy on Halloween" rule.

** I mean better in the sense of aiding their survival and banks' shareholders. They might not have been better in the sense of maximizing bankers' pay - which is of course the only purpose banks have.

Comments

[Tim Cowlishaw]

Aha! This sounds similar perhaps to the Bias-Variance tradeoff in Machine Learning?

[Tim Cowlishaw]

Aha - here's the link I tried to put in the post above but for some reason it was swallowed up: https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff

[Luis Enrique]

well I agree that it all ought to come down to better practical decision making, but there's a division of labour involved in that process. You could say that some economists do the job of exploring theoretical territory and generating models, new ways of thinking about things, that are little more than interesting stories to chew over and think about how they relate to data, other economists might produce empirical results that might not come with sufficient theory to guide policy*. Trying to synthesise all that into policy relevant stuff is not something every economist need do. If you ask me, a lot of objections to economics come from people seizing upon economics of the exploring theoretical territory kind and accusing it of failing to be something else. The example that always comes to my mind of that was mr unlearning economics tearing into a paper by Eggerston because it was not how he thought the great recession happened, whereas Eggerston wasn't trying to give an account of the great recession, he merely put forward a model in which a certain mechanism was at work which might be part of the story. Mapping theoretical territory. I am sure some economists are guilty of confusing their numerical parables for satisfactory descriptions of the world, but most of my acquaintance don't need telling.

* this is a good blog on need for theory to guide policy

https://afinetheorem.wordpress.com/2015/10/12/angus-deaton-2015-nobel-winner-a-prize-for-structural-analysis/

[Bob]

"This highlights an important and overlooked trade-off - between optimization and resilience"

That's been Bernard Leitaer's work for many years.

Chris, any thoughts on this?

http://www.3spoken.co.uk/2015/11/why-economists-fail-at-foreign.html

Why economists fail at foreign trade and why their models are wrong.

The correct approach is for each nation to operate as a currency zone free floating against each other - and with the currency managed based upon understanding how to manage floating currencies. Rather than what we currently have - fixed exchange rate thinking running currency zones into the ground.

The problem Post Keynesians have is that they really don't understand how international trade works in the context of currency zones and are misguided by working in a framework of domestic and external sectors.

Treating the 'Rest of the World' as a homogenous whole in a model of a single country economy is a recipe for disaster. You have to model the whole world as a closed system with interacting currency zone. Otherwise you miss the competition and closure feedback loops.

For example in Russia you are getting 'dollarisation' where people are giving up their rouble savings to hold USD. That expands the USD currency area into Russia and shrinks the Rouble currency area. And it is also the supply of those roubles the shorters need to cover their delivery positions.

You have the agri business in Argentina and the Petro industry elsewhere.

If you take a currency zone view, you'd quickly realise that the majority of the resource industry is outside your currency zone. To get it back in you have to tax it! Tax the pipelines! It is just as much 'outside the country' as Wolverhampton.

[Bob]

The other thing is IMF pushed 'export led growth' BS. You get these oligarchies. Insights from MMT show that exports are a real cost and imports are a real benefit. The sanctions in place are stopping imports coming into the country. So there is no need for the exports. Russia should just turn the taps off, which would bring Europe to its knees in a second. That they haven't done that suggests that Russia has the same problem as Argentina - the country is actually run by an Oligarchy that prefers to operate their export industry mostly in a foreign currency. So actually you can discount the entire oil and gas industry as a different country almost - Gasistan - operating largely outside the Russian currency area.

[Nanikore]

An unhelpful post which will no doubt make a few people feel smug.

I don't no whether this bothers you, but Keynes would disagree with you. (That also doesn't bother Krugman, apparently - if that makes you feel better).

THe problem with models is that it distracts us from engaging with the real world and real world knowledge. We have to accept the complexity, not abstract from it. You have to say "from what we can now, this is what we do know". You have to accept contradiction, in fact bring them out if the state of our knowledge has not yet got an answer. You don't make up stories to get a model.

Keynes criticised 'pretty and polite techniques'. He was referring to classical economics, but the EMH takes these things to new levels of absurdity. Economics failed to see the financial crisis coming because of models. If it was more historically engaged, took more interest in other areas like psychology, it would have been more likely to see the warning signs. Really it is just an excuse to use mathematics; for some reason economics attracts maths nerds, not people from the humanities.

Economists like everyone else should work like an investigator: you build up the story from the facts - qualitative and quantitative, and try and put the pieces of the jigsaw together. No preconceptions, no models (except for forecasting which is done after the investigation.) We don't use models for foreign policy or criminal investigations do we?

Keynes:

"[Ricardian theory], being based on so flimsy a foundation, it is subject to sudden and violent changes. The practice of calmness and immobility, of certainty and security, suddenly breaks down. New fears and hopes will, without warning, take charge of human conduct. The forces of disillusion may suddenly impose a new conventional basis of valuation. All these pretty, polite techniques, made for a well-panelled Board Room and a nicely regulated market, are liable to collapse. At all times the vague panic fears and equally vague and unreasoned hopes are not really lulled, and lie but a little way below the surface (Keynes 1937: 215).

[Nanikore]

In fact Chris you are downright contradicting yourself. If anything LTCM and the Goldman Sachs case shows exactly why you do not use models. In fact that is part of the point that Keynes is making.

I have mentioned this post to Lars Syll, hopefully he responds to it.

NK.

[reason]

I want to know who "some guy" is. It seems he is pretty smart (by the way this is a general argument against fundamentalists of all stripes).

[Nanikore]

One last one

"In fact, in a complex world, there is a positive danger in seeking the truth."

I really hope this comes back to haunt you. It is basically an excuse for people at Chicago and MIT to keep their heads in the hands - which they have been doing since 1948. "Reality..not for us!

[macmillan]

I fully agree with the sentiments expressed here. The output from models provides a basis for discussion, not something that should be accepted as the full answer. At the end of the day, no matter how much complexity is built in, the parameters in models may no longer be representative and many things may be left out.

As a former boss of mine liked to say, "some numbers beat no numbers", and model outputs should be seen in this light.

[Luis Enrique]

Maybe it would be better to say that in a complex world, there is a positive danger in seeking the complete truth

nanikore I can guess how you will see this, but I cannot conceive of how to do economics without models. I mean, you need to put numbers on things, and models are just a way of writing down what you are saying in words with numbers. I think you have model in mind, whether you try to write it down in maths or not, and in most cases it makes sense to write it down.

[Nanikore]

Luis it is a big debate, and I think the best reposte was Habermas who explained the limitations of positivist methodology in the social sciences.

OK, just say the cause of a financial crisis was nervousness about the major powers being able to control Islamic State (or think of how 9/11 affected world financial markets)

Are you telling me that we should put this into a mathematical model?

And what do these models actually tell us? Take the liquidity trap. In the intewar period an historical investigation found that the cause of this was deflationary psychology in evidence in the records of lenders, borrowers and corporations - and I find the cause of what this deflationary psychology was. I dont need a model, I summarize the evidence. I have got the causal link, I am sure about it, it is based on evidence. The ISLM (or worse) model is utterly uninteresting because it tells me nothing about causation. It just says a line shifts right and I get Y and no i effects. I can use a model to forecast, but that is not what I am doing here, I am trying to understand what is the causal link. I do not need to make up stories to get a mathematical model. Because I do not need a mathematical model. When it comes to forecasts, you simply need a basic partial regression equation (and maybe not even that) and not internal consistency and stories to get it - on that point Keynes and Friedman actually agreed.

Mathematical models can be used to give us some numerical values on things, but they should not, and cannot, be used to explain things.The only time I can think of when this is justified is when we cannot get the evidence (physicists trying to understand the origin of the universe). But note that as soon as physicists find evidence which counters their theories, they are abandoned.

[aragon]

Heuristic is the word you are looking for!

https://en.wikipedia.org/wiki/Heuristic_%28computer_science%29

"The objective of a heuristic is to produce a solution in a reasonable time frame that is good enough for solving the problem at hand."

"Results about NP-hardness in theoretical computer science make heuristics the only viable option for a variety of complex optimization problems that need to be routinely solved in real-world applications."

www.dangoldstein.com/papers/FastFrugalPsychReview.pdf

Reasoning the Fast and Fugal Way: Models of Bounded Rationality.

"Humans and animals make inferences about the world under limited time and knowledge. In contrast, many models of rational inference treat the mind as a Laplacean Demon, equipped with unlimited time, knowledge, and computational might."

"By computer simulation, the authors held a competition between the satisficing "Take The Best" algorithm and various "rational" inference procedures (e.g., multiple regression). The Take The Best algorithm matched or outperformed all competitors in inferential speed and accuracy. This result is an existence proof that cognitive mechanisms capable of successful performance in the real world do not need to satisfy the classical norms of rational inference."

Economists despair...

[Luis Enrique]

"Mathematical models can be used to give us some numerical values on things, but they should not, and cannot, be used to explain things"

that's incomprehensible to me. you might as well write you cannot use words to explain things.

[aragon]

"There are at least two ways of thinking about "what works." One is to work out "what works" and promote it through all available means-and ensure that more people do it. The other presumes that "what works" is often the product of context, conditions, and circumstance, and therefore we should be hugely cautious about the ability to understand the world or make it conform to our desires"

Fredrick M Hess - Thinking about what works

[aragon]

"the empirical fact that momentum stocks beat the market."

I would suggest that the reason for this is that the counter party (in extremis) the state, is not willing so see large numbers of investors, lemming like, disappear over the cliff. So the state steps in to keep the stock market/housing market afloat. Q.E and the rescue of the banks, AIG etc, is just the latest example of this.

EMH, is only as good as the Greenspan Put or moral hazard.

https://en.wikipedia.org/wiki/Greenspan_put

"Joseph Stiglitz criticized the put as privatizing profits and socializing losses and implicates it in inflating a speculative bubble in the lead-up to the 2008 financial crisis."

[aragon]

Luis.

"Mathematical models can be used to give us some numerical values on things, but they should not, and cannot, be used to explain things"

Correlation does not imply causation.

https://en.wikipedia.org/wiki/Correlation_does_not_imply_causation

Words don't explain things they merely communicate ideas and facts, the same with mathematics, the causal relationships behind the model attempts to explain things, i.e the economic theory.

[Nanikore]

Luis, say I want to expain the causes of WWI. I identify one cause as being a multi-polar geopolitical structure. To explain that I need to present the evidence - documented material with the facts. A mathematical model, say a game theoretic model will explain it. Maybe a knife and fork also will. But that is irrelevant. The explanation I want is the one that comes from an investigation that reaches its conclusion by puttingtogether the pieces from documented evidence (eg primary historical evidence found in government archives).

[aragon]

Of course that should be momentum investment is only as good as the Greenspan put.

Case for the defense:
http://www.economist.com/blogs/freeexchange/2009/04/in_defense_of_copula

And the EMH is only as good as the information the market (thinks) it has. Black-Scholes is predicated on the EMH.

It wasn't the correlation (Gaussian Coppola) that was the problem, but the fact that people believed it was infallible, and not the product of Government policy?

The Greenspan put
The budget surplus
Fannie Mae
Freddie Mac
Rating Agencies
Investment Banks leverage
AIG

[Luis Enrique]

Nanikore,

OK, so that's one-off historical events. But what if you want to explain variations in unemployment across countries, or think about what impact variations in the relative price of capital across countries has on the level of investment? I think you need to write down a model in maths that incorporates some mechanisms, enables you to say something about what determines the magnitude of the effects of those mechanisms, gather data on all the bits and pieces your model uses, think through what your model predicts and try to check that in as many ways as you can against what you observe. And of course you won't get a complete fit along all dimensions, you need to think about which 'errors' are fatal to your explanation, and which aren't.

(This thing that people throw around about how real scientists throw way models that are rejected by the data but economists keep tinkering with them to me just reveals how little idea people have about how much you always need to omit from any one model and hence how any one mechanisms might still be true and important but not fit the data because you haven't yet included all the right pieces)

[Nanikore]

Luis

Take variations in unemployment across countries.

Again you have to do it ground up on a case by case basis. In Germany it may have something to do with a centralised wage fixation system in place. It might share that with Japan which for historical reasons has a similar system (again the history is important to understanding why things exist). Workers in systems with firm based unions accept wage reductions in exchange for job security. Why? You need to go out and do the fieldwork to find out. Why do these systems work highly successfully in Japan and Germany but have failed in the UK? Again you have to go out on the field and find out. You need to consult sociologists, anthropologists, historians, psychologists and others who have already gone out to the factories and gathered the information and know something about it. Rational choice and other mathematical models do not and some times cannot give the us the answers. How will your mathematical model tell us anything about these mechanisms - why they exist in certain places and why they work in some places and not in others? Even if it did you are going to have to introduce artificial construct and nonsense so you can. Why don't you instead just go out and find out why?

Economics will make progress when people stop saying "what model can explain this" and start asking "what actually goes on". I want to see economists out in the field and down in the archives.

[Nanikore]

Lars did get back after I pointed out Chris's post in the commentary over there.

https://larspsyll.wordpress.com/

[Luis Enrique]

Nanikore,

I'm all for field research, but it's not going to come close to providing a satisfactory explanation. Take the union negotiated wage reductions in Germany. We're not so interested in the motivations of those who negotiated them, but whether and how that reduced unemployment in Germany. Would it work anywhere, or did it rely on Germany's unusual place as one of world's few exporters of capital goods? Does increasing job security always lower unemployment or are there circumstances in which it might raise it (perhaps by inhibiting job creation). Why didn't lower workers' wages depress demand? Is the mechanism of lower wages on unemployment, however it works, powerful enough to explain the entire difference between Germany and elsewhere, or do we need to look at other factors too? You cannot find out everything you need to know about how economies work just by going out and asking people.

[Nanikore]

They are all important questions you raise Luis. Read my post again and the question about whether something like the German bargaining system would work elsewhere is essentially the sort of question I raise. It requires knowledge about Germany and elsewhere. That almost certainly you cannot find out from a mathematical model. You need to understand things like the society and history of the country. You also need to understand how things like their production and other methods work. A lot will be gained by actually talking to managers, employees, unions, bankers etc. Going to union meetings and seeing how they work is another way. These are some ways you will find out how and why things are done they way they are. (A lot of this work has been done by historians and sociologists etc - this is information you can use for your own understanding.)

Whether we are looking for a cure for a disease, the determinants of a person's personality, the causes of conflict, we are dealing with complex interrelated factors. To address them and understand them we need to look at what actually goes on. In the end you summarize your findings. What you should not do is make up stories to get a mathematical model - your objective should be to understand what actually goes on. (Sometimes you don't need to understand what goes on to get a policy solution- but that is rare; usually to deal with something you have to know the crucial things about it.) Unfortunately there is too abstraction and story-telling in macro-economics. And economics, especially macro-economics is an anomaly. Most other social scientists gave up on positivist approaches by the 1970s. There is some in political science, but that subject area is eclectic in its approach - eg some game theory is used, but it is not the central part of explanation. By far describing what happened based on observed fact is the normal methodology and takes up by far most of the discussion.

[Egmont Kakarot-Handtke]

ICYMI

For some fatal errors about economic theory and models see ‘How economists became the scientific laughing stock’

http://axecorg.blogspot.de/2015/12/how-economists-became-scientific.html

or on the Real-World Economics Review Blog

https://rwer.wordpress.com/2015/12/10/the-blatant-absence-of-empirical-fit-of-macroeconomic-models/

[Egmont Kakarot-Handtke]

Methodological kindergarten
Comment on Nanikore of Dec 9

You say “Keynes criticised ‘pretty and polite techniques’.” And rightly so, indeed, but Keynes and the After-Keynesians in turn have to be criticized for not having a firm grasp of scientific methodology. To be more precise, Keynes was a political economist and not a theoretical economist. The political economist is an agenda pusher and interested in theory only so far as it serves his agenda.*

“I consider that Keynes had no real grasp of formal economic theorizing (and also disliked it), and that he consequently left many gaping holes in his theory.” (Hahn, 1982, pp. x-xi)

The largest hole in Keynesianism is that the profit theory is false.** And it should be beyond the slightest doubt that if one gets the pivotal concept of economics wrong all the rest of one’s theory is for the birds. The methodological incompetence of After-Keynesians is documented by the fact that they did not spot Keynes’s fatal logical blunder until this day (2011). From this follows that Keynesian policy proposals have no sound theoretical foundation. Keynesianism is commonsensical storytelling and that is enough for agenda pushing.

Keynesians insist that ontological uncertainty has been Keynes’s profound methodological insight. Think twice, this is blown up taxi-driver wisdom and the famous “We simply do not know” while obviously true is not exactly an interesting contribution to scientific progress. To the contrary, this correct self-description of Keynesians gives them sufficient reason to look for a more productive occupation.

With regard to methodology you say also that ‘Keynes and Friedman actually agreed’. This proves nothing. Both, Keynes and Friedman were political economists and what they uttered about methodology is either junk or trivial. You can be quite sure that neither of the two will ever be accepted as scientist -- except perhaps by the delusional members of their respective sects.

To quote Keynes on scientific matters or to reiterate the silly Keynesian slogan ‘Better roughly right than precisely wrong’ is self-disqualifying.

Egmont Kakarot-Handtke

References
Hahn, F. H. (1982). Money and Inflation. Oxford: Blackwell.
Kakarot-Handtke, E. (2011). Why Post Keynesianism is Not Yet a Science. SSRN
Working Paper Series, 1966438: 1–20. URL http://ssrn.com/abstract=1966438

* For details see ‘How economists became the scientific laughing stock’
http://axecorg.blogspot.de/2015/12/how-economists-became-scientific.html
** See ‘Fundamentally flawed’
http://axecorg.blogspot.de/2015/10/fundamentally-flawed.html