Forecasting vs explaining

There's a big difference between forecasting and explaining. This is one point which I fear is being under-emphasized in the culture war about the state of macroeconomics.

Critics often infer that the discipline failed because it didn't foresee the crisis.

It's certainly the case that few people saw the crisis coming. For example, only one forecaster of those surveyed by the Treasury in March 2008 foresaw a fall in GDP in 2009 - and he had also, wrongly, forecast weak growth in 2005 and 2006. UK economists weren't alone here: US forecasters also failed to see the recession coming, as did those in other countries.

Nor even was the failure confined to the Great Recession. Back in 2000, the IMF's Prakash Loungani wrote that forecasters' failure to foresee recessions was "virtually unblemished."

This tells us that the failure to predict recessions was not confined to DSGE models. Forecasters use a variety of methods, including a heavy dose of judgment. Pretty much all these methods failed.

I'm not sure that heterodox economists did much better. Steve Keen's Debtwatch focused mainly upon the Australian economy which did not suffer a recession in 2009. And even Wynne Godley didn't do much better than others. In April 2007 he wrote (pdf) of the US:

output growth slows down almost to zero sometime between now and 2008 and then recovers toward 3 percent or thereabouts in 2009–10.

That's better than most, but still some way off.

Does this near-universal failure tell us that macroeconomics is useless? Maybe not. Perhaps there's another explanation. Maybe recessions are just not predictable at all. The 2009 recession originated, to a large extent, in a micro failure, not a macro one - the collapse of banks. This is consistent with Xavier Gabaix's vew that recessions have granular origins and with Acemoglu's theory that they are propagated by network effects.It could be that the economy is so complex that forecasting is, as David says, just a mug's game*.

Many of macro's critics are begging the question: they are assuming that the economy could be predictable, if only we had a good enough theory. I doubt this**.

Now, this is just a hypothesis - albeit one consistent with lots of evidence. If you want to show I'm wrong, point me to some forecaster who foresaw both the recession of 2008-09 and the growth either side thereof. Or, failing that, show me forecasts for future years which successfully predict both growth and recessions.

It's not good enough to say "macro failed to foresee the crisis, therefore we need a new theory." Come up with the theory, and test it against the data.

Nor is it good enough to simply point to the risk of recession. Forecasting a recession that doesn't occur is as bad as not forecasting one that does; both lead policy-makers to make bad policy and investors to make bad asset allocation decisions.

We should not, however, infer that just because macroeconomic forecasting is impossible (at least, given the present state of our knowledge) that we are at the mercy of crises. At least two pieces of economic knowledge would have protected investors from the 2008 crisis, both of which were available at the time:

- the tendency for foreign buying of US equities to predict annual returns: this hit a record in 2007, warning us of trouble.

 - the seasonality of the stock market. "Sell in May, buy on Halloween" would have gotten you out of the market well before Lehman's collapsed - albeit at the price of missing out on some of 2009's recovery.

Of course, these two hypotheses told us nothing about banks, debt, real GDP or monetary policy. But this tells us two things: that economics can be useful even if macro is not; and that there's a big difference between explanation and prediction. As Jon Elster said:

Sometimes we can explain without being able to predict, and sometimes predict without being able to explain. (Nuts and Bolts for the Social Sciences, p8)

Macroeconomics - when done well, which is only some of the time - does the former. This might be all that can be expected.Whatever else is wrong with macro theory, the inability to foresee recessions is not the problem.

* Macroeconomics is not unique here: Gene points out that, for everyday purposes, the natural sciences lack predictive power too.

** I don't think we have good enough data either, but that's another story.

Comments

[Benoit Essiambre]

An explanation is only as good as the predictions it allows you to make.

That is the metric used to determine the value of explanations or models in science.

[Ivy]

Are you mansplaining or manforecasting or manjournalising?

[DTM]

Does anyone actually "sell in May and go away"? Should I actually consider shouldering the opportunity costs and transaction costs of flogging all my holdings in a couple of weeks, and then leave them largely idle over the summer?

I've heard the aphorism many times, but never actually met anyone who follows it as a matter of course.

[Luis Enrique]

"Nor is it good enough to simply point to the risk of recession"

well in this case even that would have been an improvement. At least mainstream econ does draw attention to risks of public finance or currency crises, other than a few voices*, the mainstream econ policy nexus was blind to risks in financial system

* this one doesn't get enough publicity: http://www.econ.nyu.edu/user/galed/papers.html

[Socialism In One Bedroom]

They know the Sun is going to go supernova but they narrow it down to the odd million years.

Anyway how does this fit in with the Marxist view that capitalism is a system where periodic crises are inevitable because of its anarchic nature etc?

Isn't it enough to demonstrate this?

...and then say you can't exactly predict the day of the crisis because the crisis is caused by anarchy and anarchy by its nature can't exactly be pinpointed to the exact year.

In which case the ultimate answer to the periodic crisis problem is to develop a system relatively free of anarchy?

[James]

What does making a prediction with a model actually mean? It means if you change a set of inputs, your model reacts in a way consistent with the system you're modelling.

Simple macro models give predictions about what happens to the economy when it undergoes a change in supply or demand. Given the large demand shock of 2008, most serious models have done well to predict what would happen after that shock.

In an ideal world you could incorporate complicated models of the banking sector (presumably including shadow banking etc) to predict *when* demand will collapse due to a financial crisis, but as you say, this is never realistically going to happen.

What is actually useful is being able to say what we should do in response to these shocks, and macro models have been good at that. It would be nice if we could understand how likely (stochastically) a banking sector induced demand shock might be, and the magnitude of the shock that comes from what kind of financial crisis, but this is not that important if we have a reliable policy response for the envelope of possible shocks.

The hard drive in your computer can die at any time. Do you spend a long time trying to work out when exactly it will die, or do you just back up?

[theOnlySanePersonOnPlanetEarth]

"In an ideal world you could incorporate complicated models of the banking sector (presumably including shadow banking etc) to predict *when* demand will collapse due to a financial crisis"

WTF? You have reminded us all that having a model that is relevant and logical is required before you can ascertain if the models predictions are useful or not. Because your conception, even as a casual example is off the chart stupid.

[James]

I'm not sure what your specific criticism of what I wrote is. The recession was precipitated by a financial crisis. In order to predict the recession one would have to predict when the financial crisis was going to happen and understand how this would affect the wider economy. My point was that understanding these things would be desirable in an ideal world, but perhaps not achievable in the real world, and for all practical purposes irrelevant if we actually used the appropriate policy response to crisis, however it came about.

[chris]

@ Benoit - there's sharp distinction between conditional and unconditional forecasts. Explanations and theories can be tested by the conditional forecasts they generate - eg other things equal, a higher price will reduce demand. But economic forecasts are unconditional.
@ DTM - I do thru my pension fund. I can shift costlessly between cash and tracker funds. Otherwise, it can be more bother than it's worth.

[Frances Coppola (@Frances_Coppola)]

I am a bit miffed. I thought I made it clear in the piece that appears to have started this storm that my problem is not that macroeconomics failed to predict the crisis, it is that people thought it should have done. And that some macroeconomists apparently still think that if we get policy settings exactly right, extreme tail risk events will never happen. This despite the considerable evidence that economists are fallible, their models imperfect and their knowledge incomplete - all of which can be summed up by saying that economists are only human.

[Deviation From The Mean]

"all of which can be summed up by saying that economists are only human."

Science gets better and better at forecasting, the more it discovers the more it can predict. For example, weather forecasters are only human but develop better forecasting methods and tools all the time (no really). So it isn't being human that is the problem but economics itself.

So the question is why has economics not seen such improvements? I think this is something partly to do with the fact that economics does not proceed along scientific lines but more along ideological lines. Neo classical economics is an ideological blind alley.

* I accept that economics is based on human relations, and countless variables so I am not suggesting forecasting in economics is an easy thing.

[Metatone]

I've had regular disagreements with Luis Enrique in these comment sections over the years, but I have to say that today he has absolutely nailed the issue. It's not the failure to forecast the crisis, it was the repeated claims that such a crisis was impossible. All the defenders of macroeconomics seem inclined to pretend those claims were not made, but they were.

The inability to conceive of the crisis had two crucial effects:

1) It meant we dismantled all sorts of policies that may have ameliorated the crisis, being told that they were "inefficient", "distortionary" and "anti-market."

2) It absolutely coloured responses to the crisis, whereby as soon as the peak crisis past, the tools used to get through that peak (largely Keynesian ones) were put back in the cupboard and suddenly we're back to the old policies - pretending that it couldn't happen again...

[Martin Connelly]

But Gene (who ever he is) is hugely wrong. Proper science can predict a considerable amount of things in every day life. Possibly more important, however, a scientist would not try to predict things that cannot be predicted. Economists appear to have no such qualms - thus the fun we get when we relook at their past failed predictions.

Would be interested in a few of your examples showing Macro being well done, because they appear few and far between???

[Keith]

What ever the detailed models with their maths say the bulk of the economic and financial industry even more so do give the impression of believing like the white star line that their ship HMS Titanic cannot sink as it is so big and modern. But no one told the ice burg. The crass smugness of lots of very rich people is amazing to behold. Unsinkable ship/ perfect economics which has solved all problems...

Equally odd and scandalous is the response to the fuck up in finance which mostly involves deranged cuts to public sector spending bearing no relation to the precipitating cause.

[Winslow R.]

"Critics often infer that the discipline failed because it didn't foresee the crisis."

Keynes covered this turf when he said something like, 'when the facts change, I change my mind'.

We find economics to be useless because it somehow misses that facts are not random or predictable but are instead determined by policy makers thinking that they acting in their own interest.

[Min]

On the one hand we have the idea that macroeconomics is not a predictive science, and on the other hand we have economic projections 75 years into the future. Tilt!

[rogerh]

In principle, what would it take to control banks to avoid 'blow ups'. Well you don't run a chemical plant by looking at the annual report - it can explode in a millisecond, similarly a bank can blow up in a few weeks. So if you want to avoid banking blow-ups you might decide to micro-monitor (and understand) their every move (all of them). This seems a big job with a big moral hazard attached and expensive too. In the end who is going to pull the plug - for chemical plants and banks are excellently productive and convenient right up to the last moment. Which leaves the economists to pore over the data logs looking for why the bang occurred whilst everyone else cleans up the mess. All that tells you is why this particular bang occurred - which suggests that either the occasional bank blow-up is unavoidable or the bankers must face more severe hazards if they screw up (to encourage the others).

[reason]

Look,
MANY people in the period leading leading up to 2008 that the RISKS of a MAJOR crisis were rising. And I don't agree that the causes of the crisis were exogenous, it was an endogenous problems as trends (accelerating asset price increases, increasing debt levels, falling yields, large external deficits) that couldn't possibly continue were in evidence.
But yes predicting precisely what would happen when was as difficult as making the same prediction for an earthquake or volcanic eruption.

[chris]

@ Metatone, Luis. Yes, some said that crises were impossible, and had models consistent with that view. But that was just lousy economics; it shouldn't discredit the whole macro profession.
@ Reason - yes, some were warning of that. But for practical purposes you could have inferred that from the high foreign buying of US equities. You didn't need a good explanatory theory to have done so.

[masaccio]

I think macroeconomics is a failure because the policy prescriptions of its practitioners caused the Great Crash. Based on the ideas of actual economists, proud of their insights and theories, the financial sector demanded that regulatory structures be dismantled, urged borrowing as a plausible behavior for paying daily expenses, and set up unmanageable business structures that were rife with fraud, lying, and cheating. People who were getting richer got their pocket politicians to support this program. I'm sure you all know the names of these people.

One possibility is that economists designed the policies to please their sponsors and benefactors. One possibility is that economists were stupid. In either case, they were horribly wrong, and caused terrible damage.

That's why people loathe the stiff-necked fools who insist that they were right and are unable to acknowledge their failures and change their behavior. Fools who continue to insist they were right about their policies, and continue to demand that we put them into practice, causing further damage.

[Ari Andricopoulos]

1) That "Sell in May, buy on Halloween" worked is not a valid defence of macroeconomics. In fact, if everyone had done it, the crash would have been in May instead.

2) The issue isn't that economists should be able to predict exactly when a crash will come. The issue is that there has been a huge build up of private sector debt and it was for the most part ignored. Minsky didn't ignore it but his ideas were ignored.

Any model of economics has to include the dynamics of private sector debt. This really should be obvious by now.

[J Stri]

Looking to economists and the mainstream financial "experts" won't help you mitigate the risk of your portfolio. Economists ( and fund managers ) aren't specialists in tactical asset allocation or computational financial engineers. Because of they are paid salaries, they don't have incentive to do "deep" research into risk management, as would a private entity with their own "skin in the game" and they basically follow staus quo thinking.
As you have brought up "Sell in May, buy in Halloween", this seasonality has been accurate in producing alpha in the small cap value universe when used with a robust, cutting edge, and modern tactical allocation model. https://docs.google.com/presentation/d/1C37CJypoxHWHB09e3g25ewOGjP83wDZhj5j6tlrLJoA/edit?usp=sharing

[Metatone]

@Chris - it's a reason to discredit macroeconomists as a tribe if they don't take action against those who are practicing lousy economics, instead of lauding them and continuing to treat them as important figures in the field.

(And of course, crucially, raising not a whimper, sometimes even defending them while they give duff advice to politicians about the economy...)

We end up with "media macro" (as SWL has coined it) mostly because macroeconomists failed in their first duty as a profession, policing the quality of their field and taking on lousy practitioners, even when they have attained some power, status and influence...

[Ari Andricopoulos]

One more thing; dismissing Steve Keen's predictions because they were based on the Australian economy is extremely unfair. The reason Australia didn't have a recession was because of China's massive stimulus.

His point was that private debt had reached unmanageable levels. And he was right.

Incidentally, Australia's debt problem continues to grow and my prediction is that all he got wrong was the timing.

[Gene Callahan]

"But Gene (who ever he is) is hugely wrong. Proper science can predict a considerable amount of things in every day life."

I actually gave examples of many things physics can't predict, that yet follow the laws pf physics in some sense. You've given no counter-examples, just said "hugely wrong."

You might see Nancy Cartwright's _Why the Laws of Physics Lie_ for one of the most prominent philosophers of science showing this fact at length.

[Gene Callahan]

"An explanation is only as good as the predictions it allows you to make."

Nonsense, Benoit: evolutionary theory cannot predict what species will arise in 100,000 years at all.

Nevertheless, it offers great explanations for many things.