The Economist makes a claim which, though common, I find implausible:
Almost as damaging is the hash that banks have made of “value-at-risk” (VAR) calculations, a measure of the potential losses of a portfolio…The mistake was to turn a blind eye to what is known as “tail risk”…In markets extreme events are surprisingly common—their tails are “fat”.
I just can’t believe banks did this as an intellectual error, because it's easy to incorporate tail risk into VAR. This is because we do have a statistical description of tail risk - the power law.
To see how this affects VAR, let’s take a very simple example, starting by assuming a normal distribution.
You have a £100m equity portfolio with annualized volatility of 30%. You want to know what daily VAR is at a 1% probability.
All we do is deannualize that 30%, which gives us daily volatility of 1.86% - this is one standard deviation. We then read off from a normal distribution table that a 1% probability is a 2.32 SD event.
So we multiply 1.86 by 2.32.
This tells us that VAR is £4.32m. We can expect to lose £4.32m or more in one day every hundred.
To incorporate tail risk, we just have to tweak this.
We do so by recognizing that losses of more than two standard deviations are distributed as a cubic power law. This means that, to find the probability of a rare event - with SD of x - we simply divide x by two and then raise to the power 3. This tells us how much less likely the event is than a two standard deviation event. So, for example, a 10 standard deviation loss is five to the power three = 125 times less likely than a two standard deviation event.
Now, we are interested in 1% probabilities, which are 2.28 times less likely than a two SD event. Our cubic power law tells us these are 2.63 SD events. So, we just multiply 2.63 by 1.86. This gives us £4.89m.
So VAR is 13% greater than under a normal distribution.
This doesn’t seem much of a difference. But this is because 1% events are quite common. It’s when we go further down the extreme of the distribution that tail risk matters. For example, for one day in 1000 losses, a normal distribution gives us a VAR of £5.75m, but our cubic power law gives us a VAR of £10.55m.
And here’s my problem. Any idiot can tweak VAR models to incorporate fat tails - hey, an idiot just did. I just can’t believe that banks’ risk managers have been so stupid as to be unable to do this. It’s certainly not the case that fat tails are something new; we’ve known about them since at least October 1987.
Instead, I suspect that banks’ failure lay elsewhere. One difficulty was that their portfolios were much more complex than mere equities, so it was harder to calculate the distribution of extreme returns. But I would have thought that, given that these portfolios were vulnerable to default risk, liquidity risk and correlation risk, it would have been reasonable to assume an equity-like cubic power law, or an even higher exponent.
Another difficulty, though, was internal politics. Risk managers might have wanted to assume fat tails and a high VAR, but traders, anxious for a freer rein, would have resisted.
Which comes down to an important point. Banks’ failures are more likely to have been due to organizational failures - the difficulty of measuring their portfolio and reconciling competing internal interests - than to the inability to use mathematical models. Which in turn suggests they need more radical reform than merely better risk models.
To see how this affects VAR, let’s take a very simple example, starting by assuming a normal distribution.
You have a £100m equity portfolio with annualized volatility of 30%. You want to know what daily VAR is at a 1% probability.
All we do is deannualize that 30%, which gives us daily volatility of 1.86% - this is one standard deviation. We then read off from a normal distribution table that a 1% probability is a 2.32 SD event.
So we multiply 1.86 by 2.32.
This tells us that VAR is £4.32m. We can expect to lose £4.32m or more in one day every hundred.
To incorporate tail risk, we just have to tweak this.
We do so by recognizing that losses of more than two standard deviations are distributed as a cubic power law. This means that, to find the probability of a rare event - with SD of x - we simply divide x by two and then raise to the power 3. This tells us how much less likely the event is than a two standard deviation event. So, for example, a 10 standard deviation loss is five to the power three = 125 times less likely than a two standard deviation event.
Now, we are interested in 1% probabilities, which are 2.28 times less likely than a two SD event. Our cubic power law tells us these are 2.63 SD events. So, we just multiply 2.63 by 1.86. This gives us £4.89m.
So VAR is 13% greater than under a normal distribution.
This doesn’t seem much of a difference. But this is because 1% events are quite common. It’s when we go further down the extreme of the distribution that tail risk matters. For example, for one day in 1000 losses, a normal distribution gives us a VAR of £5.75m, but our cubic power law gives us a VAR of £10.55m.
And here’s my problem. Any idiot can tweak VAR models to incorporate fat tails - hey, an idiot just did. I just can’t believe that banks’ risk managers have been so stupid as to be unable to do this. It’s certainly not the case that fat tails are something new; we’ve known about them since at least October 1987.
Instead, I suspect that banks’ failure lay elsewhere. One difficulty was that their portfolios were much more complex than mere equities, so it was harder to calculate the distribution of extreme returns. But I would have thought that, given that these portfolios were vulnerable to default risk, liquidity risk and correlation risk, it would have been reasonable to assume an equity-like cubic power law, or an even higher exponent.
Another difficulty, though, was internal politics. Risk managers might have wanted to assume fat tails and a high VAR, but traders, anxious for a freer rein, would have resisted.
Which comes down to an important point. Banks’ failures are more likely to have been due to organizational failures - the difficulty of measuring their portfolio and reconciling competing internal interests - than to the inability to use mathematical models. Which in turn suggests they need more radical reform than merely better risk models.
The problem is simple, a mountain of credit on top a mole hill of money.
Posted by: passer by | January 27, 2009 at 11:33 AM
Very well put. This is my big compaint with the Taleb viewpoint which seems to point to a lack of understanding of probability as the key issue, rather than political economy. I put this to NNT at a meeting. His reply ("yes, it's all political economy") suggests that even he accepts that it's more an organizational and incentives problem than a purely mathematical one.
Sam.
Posted by: Sam Z. | January 27, 2009 at 12:02 PM
Any fool, as you say, can manipulate a mathematical model: "recognizing that losses of more than two standard deviations are distributed as a cubic power law....". The trick is to find a mathematical model that represents reality well enough.
Posted by: dearieme | January 27, 2009 at 12:54 PM
Well, quite. Risk models (and I've worked on my share) work great until the Black Swan turns up. The one that causes the problem is, pretty much by definition, the one that everyone missed. And everyone missed it because it was literally unthinkable. Or at least un-thought of. Which is more or less what NNT was going on about. The failure was to recognise that there were Black Swans lurking out there and pay/trade for "insurance" against general catastrophe. Of course, doing that in a sufficiently liquid market would reduce returns down to somewhere near risk-free.
I'm somewhat (OK, very) concerned about the way we're seeing institutions described as "too big to be allowed to fail". More so when governments force such over-big failures into mergers, creating even more too-bigness as a result. I'm not particularly impressed by economies of scale, they're usually offset by the inefficiencies of large organisations, so on balance I think there's a reasonable argument for decomposing large risk-taking companies into something small enough to be allowed to fail, and ensuring that they stay small enough.
Posted by: Mike Woodhouse | January 27, 2009 at 01:23 PM
Very interesting discussion. W
hile NNT certainly goes on and on about the evils of VaR, I think his view is incredibly nuanced - almost irritatingly so. VaR's basic problem is that it works really well almost every time; it is just the one time it does not work, it is catastrophic (black swan blindness). Humans seem to lack the ability to really internalize this. Thus, NNT has concluded we shouldn't use VaR ever.
I think he makes a great point that instead of trying to come up with ever more sophisticated mathematical models (GARCH, etc.) to more precisely understand the world, we should accept the limits of human understanding (esp. in the modern world) and set up our institutions and laws based on what we don't know vs. what we think we know (or can know).
His Edge article on the 4th quadrant is brilliant, and a great starting point.
Posted by: Russell | January 27, 2009 at 02:11 PM
The real problem is using the VAR models etc etc in the first place.
In the old days, a Bank Manager would look at a proposition and if he didn't understand it, it didn't get done. We need to get back to that kind of simplicity.
Posted by: kinglear | January 27, 2009 at 02:54 PM
If it was just down to VAR models, we wouldn't be in this mess. Over the last 15 years 70% of the growth in leverage was within the banking system itself - the evolution of the virtual banking system as Paul Mcculley of Pimco calls it. The use of market prices, principally on credit default swaps was the key error; a fall in the CDS spread was seen as an objective measure of issuer risk when it was actually a measure of investor risk appetite. WIth nobody doing the due dilligence on the issuer the system created literally thousands of new asset backed AAA products that offered high yields with "no risk". There was no risk becasue the rating said so. Then it all turned on its head and the same CDS models worked in reverse. But even this was not enough to lead us where we are now. The Fannie and Freddie conservatorship, the ban on short selling and the collapse of AIG accelerated the unwind of the CDS driven models. But the collapse of lehman killed off the ability of money market funds to buy Commercial Paper and thus enable the real world to tap the virtual banking system for working capital. So it ran out. So everyone cut output and sold inventory. So the unwind of the financial sector finally hit the real economy. VAR models didn't help but it was Hank's bazooka wot done it.
Posted by: Mark T | January 27, 2009 at 04:34 PM
To tell the truth, being the word 'models' on the title, I would have expected any of your uplifting photos.
Posted by: ortega | January 27, 2009 at 05:36 PM
Couldn't the fact that it's pretty hard to know what the SD is in the first place a problem for VaR too ?
Posted by: BG | January 27, 2009 at 10:13 PM
You're quite right. In fact the VaR models did have some of these problems, but risk managers were quite aware of them, and were capable of taking them into account. The real problems were much more gross - at least in the one failed institution whose VaR model I knew about, there was simply no serious model at all of the risk of MBSs. If you've no grounds on which to estimate the volatility, then no amount of fiddling with the distribution will tell you what the risk really is.
Posted by: A Programmer | January 27, 2009 at 10:20 PM
ortega - I couldn't agree more. Huge disappointment, and it wouldn't even have been that gratuitous.
Posted by: kinglear | January 28, 2009 at 08:33 AM
In your discussion you use the standard deviation obtained predominantly from the centre of the distribution to infer something about the tails. That is the big problem I'm afraid, you just don't have much data relating to the extreme events like one or two observations at best. Given that it is of course impossible to gauge anything statistical about them. You might be better off using your imagination to come up with catastrophes and seeing what hits your portfolio takes under those conditions. Some institutions use this method in practice. However the world always finds ways to shaft you in excess of the worst figments of your imagination!
Another big problem with VAR is that the values go down when you have a period of stability, this makes it strongly pro-cyclical. Like now you probably won't lose that much more on AAA subprime bonds but you can bet that the risk wizards would want you to have a big cushion against them.
Finally if everyone uses the same risk measures then we end up with systematic problems rather than localised ones. Searching for some unified solution is going in the wrong direction as are the efforts of ratings agencies to basically copy one anothers work. There's too much consensus throughout finance, maverick's are hardly tolerated at all.
Posted by: russ | January 29, 2009 at 08:39 AM
Taleb and Kahneman in Munich this week; Taleb of course rails on VaR. Kahneman helps keep Taleb (relatively) calm. I have never seen the two of the share the same stage - fantastic hour of conversation imho.
http://www.dld-conference.com/2009/01/reflections-on-a-crisis.php
Posted by: Russell | January 29, 2009 at 04:53 PM
" Any idiot can tweak VAR models to incorporate fat tails - hey, an idiot just did. I just can’t believe that banks’ risk managers have been so stupid as to be unable to do this."
No, far from it, it statistically and mathematically very hard !
Two main reasons : Identification of parameters on fat-tails laws from historical data is a procedure that is very unstable numerically ( a point that Taleb himself recognized). So the confidence interval for your tail parameter becomes so large that the whole VaR exercise becomes meaningless.
It becomes even worse when one has to tackle multivariate law : the set of available families of copula function is quite small, most of them tend to underestimate comovements, and parameter identification is more unstable than for univariate laws.
Posted by: Charles | February 05, 2009 at 03:05 AM
Very good post!! It's nice to read on VaR. I completely agree with you!
Posted by: Kiara | August 19, 2009 at 04:46 AM
And a lot of it reflects a switch from bank deposits to securities; foreigners “other investments” in the UK, http://www.watchgy.com/ mostly bank deposits, fell by £143.2bn in Q1. And of course there’s no guarantee such buying will continue.
http://www.watchgy.com/tag-heuer-c-24.html
http://www.watchgy.com/rolex-submariner-c-8.html
Posted by: rolex day date | December 27, 2009 at 04:44 PM