Forecasts as mirrors

Matthew Parris calls economic forecasts right:

I suspect that economic forecasting owes more to a mirror held up to yesterday than a crystal ball held up for tomorrow.

He’s bang on, for two reasons.
First, the test of an economic forecast is not: is it right? But: will it sell? The buyers of economic forecasts want to be told that they have made the right decisions. As Montagu Norman allegedly said to an early Bank economist: “your job is not to tell us what we should do, but to explain why we’ve done what we have.”
So, if a Bank has made lots of mortgage loans, it doesn’t want to hear that the housing market will collapse. And a firm that’s just invested in new capacity doesn’t want to hear there’ll be a downturn.
In some cases, these pressures are compounded by outright vested interests. You wouldn’t expect the Nationwide (say) to be the first to forecast a housing slump.
The result is that forecasts are biased towards the view that the future will resemble the recent past.
And this is quite sensible for another reason. There is a view - expressed (pdf) by Paul Ormerod - that GDP growth, among other things, is close to random. It’s genuinely unforecastable. And the best forecast of a random variable is its current value.
These two pressures mean Parris is right. Economic forecasts are mirrors. And usually, this is no bad thing, because the macroeconomy usually pootles along steadily. The problem only comes when shocks push the economy into recession. But shocks, by definition, are unforecastable. The upshot is that economists are usually caught off-guard by recession.
Forecasts, then, let folk down just when they are most needed.
But who’s the bigger fool here - the economist who is merely responding to market forecasts, or the person daft enough to risk money on the belief that the economy is predictable?

Comments

[Glenn]

'Economists' give forecasts based on complex models and then also on the hoof. Economists are expected to provide some view of the future. There's nothing so wrong with that, but the problem is the expectations of users of forecasts - they put too much trust in them.

My 10 golden rules for doing and using forecasts!

1 Forecasts should be used as part of a process of trying to understand the future – they provide only a partial insight into the economy
2 Any judgements about the future are not facts waiting to happen – there is a chance that they won’t come true
3 Any judgements about the future are based on current information – and this information can be out of date, inaccurate or wrong
4 The forecasts you use should be based on the most accurate (thanks dearieme) and up-to-date information available
5 It is strongly recommended that forecasts do not lead policy response
6 For any forecast it is correct to focus on broad movements and changes rather than on precise numbers
7 The user should recognise that all projections are subject to error
8 Forecasts cannot predict shocks or sudden events
9 Forecasts should be read with a critical eye – they are usually based on modelling techniques and can overlook the specific uniqueness of local characteristics
10 Finally, forecasts should never be used as the sole input into planning or decision-making

Who is the bigger fool!? I would say that the bigger fool is the person who places forecasts as the most influential factor in their decision making. The lesser fool is the person who provides the forecasts and pretends that they can either accurately predict the future or can predict it more accurately than anyone else.

[chris]

This is good advice.
I think it was Mervyn King who said forecasts should be regarded as guideposts - ways of measuring how suprised we should be by events.

[kinglear]

I think forecasts are a bit like Italian road signposts. They express aspirations rather than point directions.Anyway, I hope you and all your readers had taken out Black Swan insurance....

[DMP]

"And this is quite sensible for another reason. There is a view - expressed (pdf) by Paul Ormerod - that GDP growth, among other things, is close to random. It’s genuinely unforecastable. And the best forecast of a random variable is its current value."

This seems to suggest the best forecast of a random variable is its mean. (As you imply, the mean of a variable that follows a random walk is its current value.) But this isn't the case in general.

Salesman A sells goods on consignment while B sells non-returnable perishable items. Observers note that A appears to consistently overestimate demand (excess goods are returned at low cost) while B consistently underestimates demand (some potential customers are left unsatisfied since wasted goods are cost more than the profit margin - go to a bakery late and the shelves are likely to be near empty.) Both are rational despite their mean forecast errors not being zero over a large number of forecasts.

We might define the optimal forecast of a random variable X as the value Z that minimizes expected losses from making a forecast error, say e, the difference between the forecast and the realized value (X - Z). The optimal value of Z depends on the loss function L(e) and the distribution function f(x) If f(x) and L(e) are known then Z can be determined and it won't necessarily be the mean of X as in the cases of salesmen A and B.

For example if losses are linear in the error e and symmetric - so loss from the forecast error e is proportional to |e| then the optimal forecast is the median of X (and also incidentally the mean of X is f(x) is symmetric). If L(e) is quadratic in e the optimal forecast is the mean of X for any distribution function f(x). In short, situations in which the mean (or conditional mean) is the optimal forecast are special cases.

[Bob B]

Suppose we switch from the subject of economic or business forecasts to forecasts about other issues? How about these?

"Lead contamination most often arises from the dust and soil, but can also come from lead water pipes or environmental pollution. Studies have consistently found associations between lead exposure in childhood and subsequent antisocial behaviour, but there have always been problems in determining causality."
http://www.newscientist.com/article/dn13989-childhood-lead-exposure-can-predict-criminality.html

"People who smoke the strongest form of cannabis - known as skunk - may be more at risk of psychosis than those who use milder forms, UK researchers claim."
http://news.bbc.co.uk/1/hi/health/7485385.stm

Should governments really do nothing when such forecasts or statistical associations surface? The original medical case against smoking tobacco was based on statistical associations.

Personally, I regard it as sensible to go out with an umbrella or a raincoat if the weather forecasters predict rain in the near future in my region and believe it prudent to safeguard houses and cars against the possibility of theft.

[Bob B]

Another thought.

"The problem only comes when shocks push the economy into recession."

Only? What of endogenous business cycles such as can in principle be generated by interactions between the multiplier and the accelerator?

For a quick intro to the extensive historic literature, try this:
http://cepa.newschool.edu/het/essays/multacc/multintro.htm

[Bob B]

Unfortunately, M&S is not now alone in reporting falling high street sales. In today's business news:

"Evidence that shoppers are deserting the high street continued today as John Lewis said that sales in its department stores had plunged 8.3 per cent, the seventh time in eight weeks that sales have declined. . . One bright spot was the performance of John Lewis Direct, the company's internet business. The internet sales channel saw its busiest Saturday since launch last weekend, with a 20 per cent year-on-year increase in sales, indicating that even at sales time shoppers prefer to bag bargains from the comfort of their own home."
http://business.timesonline.co.uk/tol/business/industry_sectors/retailing/article4267107.ece

[jameshigham]

Economic forecasting is crystal ball stuff, by and large as each economist has part of the picture from a particular angle but not the rest of it.

Take the M and S downturn. The economists will look for economic downturns and the corporate advisers will blame poor corporate choices but the truth is that there counter service and around the store has just gone downhill.

Grumpy salespeople or lack of them trumps any economics.

[Bob B]

We'll need to maintain careful watch to see what is happening to the sales of other store chains besides M&S. The John Lewis Partnership stores were doing especially well last Christmas:

"Department store group John Lewis has reported an 8% rise in sales during Christmas week, driven by MP3 players, televisions and other electrical goods."
http://news.bbc.co.uk/1/hi/business/7171491.stm

The downturn in recent sales of that group is therefore going against previous trend.

I certainly don't accept that economic forecasting is just "crystal ball gazing" - unlike the forecasting of medics, criminologists and psychologists etc . . .

"Roger Bootle of Capital Economics had predicted a correction in the UK housing market by as much as 20%. In an article published yesterday [14 May 2006] in The Telegraph, and reproduced below, he explains why he turned out to be wrong and discusses what lies ahead for house prices."
http://www.in2perspective.com/nr/2006/05/why-the-uk-housing-market-did-not-crash.jsp

Roger Bootle may have got the timing of the downturn in the housing market wrong - for reasons he analyses - but the housing market certainly has turned down. He was right about the substantive point - the ratio between average earnings and average house prices was at unsustainable levels. And we later found out that some banks were offering 100% and even 125% mortgages. That couldn't go on. Charles Goodhart, previously of the Bank of England's monetary policy committee, was reported in the press as concerned about an emerging house price bubble as far back as 2000. And we now know by IMF stats that Britain has a relatively larger house price bubble than the US.

[Bob B]

In The Economist, out today:

"It is going to get nasty; exactly how bad depends on the Bank of England and, especially, Gordon Brown."
http://www.economist.com/opinion/displaystory.cfm?story_id=11670314

Btw if economic forecasting is just "crystal ball gazing" then what the hell is the Bank of England doing targeting inflation? The Monetary Policy Committee is basing decisions about setting interest rates on, among other factors, forecasts of inflation in Britain two years downstream . . .

[Glenn]

Well from my experience, of the better regarded forecasters, the "crystal ball" in question is a big and complicated macroeconomic (multisectoral dynamic macroeconomic) model that takes several people working full time to keep up to date and relevant. E.g Cambridge Econometrics, OEF, Experian, etc.

I know some banks, BS's and other 'economists' just use trend analysis which is less good.

Dunno about the BoE's model. They'll have one.

HMT publishes a monthly summary of forecasts for the UK economy - about 50 odd forecasters. Quite handy to look at the spread.

[Dipper]

Economics strikes me as being similar to Materials Science or engineering. A materials scientist or engineer can generally tell you all sorts of things about materials and structures, where the weak points are, and what will happen if you stress the system. But what is very difficult to predict is when something will break.

I find economists are very good at telling you the numbers to look out for, and which side is the dangerous one. As for predicting when something will collapse, and how far it will go; if they knew that, they'd be in a hedge fund investing, not telling you about it.

[Bob B]

Famously, HM Treasury staff maintain an economic forecasting model which usually attracts plaudits for the accuracy of its forecasts compared with other forecasting agencies.

The Item Club, managed by Ernst & Young, uses the Treasury model to generate its own forecasts, the difference with the output of the Treasury's use of its own model being that the Item Club has to input its own assumptions about government policy decisions:

"The UK is likely to avoid recession in 2008, experiencing only a more mild 'rebalancing' of the economy, according to forecasts by the Ernst & Young ITEM Club." [22 Jan. 08]
http://www.telegraph.co.uk/money/main.jhtml?xml=/money/2008/01/21/cnitem121.xml

Other well regarded models: Institute of Fiscal Studies (IFS) - especially the annual "Green Budget" of the IFS, meaning by Green "consultative", not necessarily "environmentalist" - and that of the National Institute of Economic and Social Research (NIESR).

Both are regularly reported on publication in the Financial Times. A significant percentage of NIESR staffers, including the Director, are usually ex-Treasury. This doesn't mean they are uncritical of official Treasury policy - often quite the opposite - but they are perhaps better informed about what goes on there. Interestingly, NIESR staffers were more enthusiastic about the UK joining the Euro than was the Treasury in the lead up to the decision based on the Treasury's five tests announced in June 2003 and it's now widely recognised that the Treasury's own assessment was better founded. Precious few mainstream economists are now cheering on the supposed benefits of Britain joining the Eurozone anytime soon. Walter Eltis, oddly enough Heseltine's economic adviser in the DTI in the 1990s, deserves special credit for putting the spotlight on the basic flaw of joining the Euro is his book: Britain, Europe and EMU (2000):

"In much of continental Europe, familes obtain finance for home ownership or personal consumption far less readily. . . In consequence, aggregate mortgage debt is 60 pc of GDP in Britain but only 40 pc in Germany, 25 pc in France and less than 10 pc in Italy. The interest rates paid on these lower levels of personal debt are also less flexible than in the UK. The variable interest liabilities of the United Kingdom personal sector total 64 pc of GDP. They are only 16 pc in France, 3 pc in Germany and 2 pc in Italy." [p.185]

Consider what would have happened to the house price bubble in Britain if interest rates here had been held at the levels prevailing in the Eurozone.

The Euro assessment is a good example of "what if" economic analysis where econometric forecasting is only a part of the assessment. A more fundamental part relates to the potential consequences of alternative economic structures - joining the Euro would have meant locking the exchange rate of the Pound against other west European currencies for an indefinite future; staying outside means retaining a flexible exchange rate for the Pound and national autonomy over monetary policy.

[Glenn]

Hi Bob - the HMT are usually praised for their accuracy, sure but I actually think its in large part because they get all the official data (incl very detailed survey returns) much sooner than others - so the 'data lag' on their forecasts is less of a problem than other forecasters.

Interestingly, I've done work in this area, and there's been interesting papers from the National Bank of Sweden and the Institute for Employment Research on where the main forecasting errors come from - and its about 80% due to base data inaccuracies.

[Bob B]

"I've done work in this area, and there's been interesting papers from the National Bank of Sweden and the Institute for Employment Research on where the main forecasting errors come from - and its about 80% due to base data inaccuracies."

If so, that is interesting and leads us back to chaos theory:

"chaos theory describes the behavior of certain dynamical systems – that is, systems whose state evolves with time – that may exhibit dynamics that are highly sensitive to initial conditions (popularly referred to as the butterfly effect)"
http://en.wikipedia.org/wiki/Chaos_theory

If initial conditions are misspecified because of data errors then the rest follows. What worries me if that if 80% of forecasting errors are due to base data errors then that doesn't leave much room for forecasting errors due to: (a) unforeseen system shocks (eg Northern Rock bank runs, externally determined oil and food prices); (b) unanticipated policy decisions whether in the HMT's forecasts or in those of outside forecasting agencies.

What shocked me was the mess we got into after formally abandoning "monetarism" in the autumn of 1985 when interest rates were set by the Chancellor and expected to achieve two potentially conflicting policy objectives: (a) internal stabilisation such as curbing inflation and (b) maintain a competitive exchange rate for the Pound on the way to joining the ERM, which we eventually did in October 1990.

The trouble was that Britain needed rather higher interest rates earlier in the later 1980s to rein back run-away economic growth but lower interest rates to keep a more "competitive" exchange rate to prevent the Pound getting locked into the ERM at an over-valued" exchange rate. Now I thought it had always been well understood since Tinbergen's classic: On The Theory of Economic Policy (1952) that each policy target needed its own policy instrument:
http://www.garfield.library.upenn.edu/classics1986/A1986C401200001.pdf

But it seems that Nigel Lawson and the Treasury overlooked that with the consequence that GDP growth reached unsustainable levels by the end of the 1980s and inflation started to surge again after we had joined the ERM.

Evidently, there's more at stake with economic policy than just the accuracy of economic forecasts. For one thing, we need structures and systems that are sufficiently resilient to withstand forecasting errors.

[Robbo]

Quote: "the best forecast of a random variable is its current value."

Really ?

goes away muttering under breath ..250 years of probability theory...Laplace...Gauss...Fisher...unbiased estimator....

[Buy Generic Sildenafil]

HI
And a firm that’s just invested in new capacity doesn’t want to hear there’ll be a downturn.

[Buy Sildenafil Citrate]

Hello
And usually, this is no bad thing, because the macroeconomy usually pootles along steadily.

[Cheap Generic Viagra]

The chaotic way that follows our economy is terrible I think...

[Online Foreign Pharmacies]

I was psyched to surf over here and see that you've recommend that post.

[Viagra Online]

Forecasts as mirrors can u believe that

[Rådgivende ingeniør]

This is a very useful tips. To forecast like a mirror it means they are simply forecasting what is the truth. It must be implemented by the other forecasting company. Like weather bureau they must be consistent because typhoon is a very bad weather that can destroy houses and even lives of the people and other living things. You have a very nice and helpful article. Keep up the excellent work!

[Buy Viagra]

hello
I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.

[Viagra Online]

Hello fellas here, Its great. But at Google Maps I just found some pics...I think you will be very pleased lol

[us drugstore]

Thanks for the review. I think it was Mervyn King who said forecasts should be regarded as guideposts - ways of measuring how suprised we should be by events.

[buy viagra]

thanks for collect and post this information about Forecasts as mirrors I'm very interesting in this blog

[Comprar Viagra]

You have a very good blog that the main thing a lot of interesting and useful!

[ps3 купить]

They're filming it seems like every day here in Atlanta. Can't wait to see it on the tube and look for familiar places...

[apotek dk]

Jeg tror, ​​det var Mervyn King, der sagde, prognoser skal betragtes som pejlemærker - metoder til at måle, hvordan overrasket vi skal af begivenhederne.