Last week I was reading a document published by my company called Dealing with the Unexpected, which gives some lessons learnt from the recent credit crisis. Early in the paper the authors speak about a 10 sigma event, or an event that we only expect to see once in 10,000 years. This piqued my interest because I remembered an infamous statement made by some financial leader during the onset of the financial crisis to the effect that we are now experiencing repeated 25 sigma events. Already a 10 sigma event is quite unlikely, but a 25 sigma event is just absurd –exponentially smaller, but just exactly how much? A one in a million year event? One in a billion years?
First, what is sigma? In statistics and probability, the lower case Greek letter sigma is used to denote the standard deviation of a distribution, which as the name implies, is the accepted unit to measure how much an outcome can vary from its mean or average. A Wikipedia article lists the following table for the likelihood of sigma deviations for the standard normal distribution
So at 6 sigma deviations we are already talking about events that occur once every 1.5 million years. Note that the scale is not linear, and the difference between 2 and 3 sigma events is less than the difference between 3 and 4 sigma events.
A 25 sigma event must be very unlikely indeed – so unlikely that the references I searched don’t even bother listing this value, including the news articles that carried the original quote. But after searching directly for sigma events I found a wonderful paper from researchers at the business school of the University College Dublin. The researchers are a little more pessimistic (by a factor of 2) in their calculations since they are only concerned with positive deviations away from the mean, as shown in the diagram below for the the 2 sigma case.
The paper reminds us that the 25 sigma quote came from David Viniar, CFO of Goldman Sachs, who actually said in the Financial Times in August 2007 that
We were seeing things that were 25-standard deviation moves, several days in a row.
Not just one 25 sigma event, but several! So the likelihood of those higher deviations was calculated to be
or 1-in-10^{135} years for a 25 sigma event. For example, this is much less likely than guessing an AES-256 key in one attempt. The researchers offer the following comparison as to how unlikely such an event is
To give a more down to earth comparison, on February 29 2008, the UK National Lottery is currently was offering a prize of £2.5m for a ticket costing £1. Assuming it to be a fair bet, the probability of winning the lottery on any given attempt is therefore 0.0000004. The probability of winning the lottery n times in a row is therefore 0.0000004^n , and the probability of a 25 sigma event is comparable to the probability of winning the lottery 21 or 22 times in a row.
Either Mr. Viniar was very confused or his models were very confused. Or potentially both. A final quote from the paper
However low the probabilities, and however frequently 25-sigma or similar events actually occur, it is always possible that Goldman’s and other institutions that experienced such losses were just unlucky – albeit to an extent that strains credibility.
But if these institutions are really that unlucky, then perhaps they shouldn’t be in the business of minding other people’s money. Of course, those who are more cynical than us might suggest an alternative explanation – namely, that Goldmans and their Ilk are simply not competent at their job. Heaven forbid!
Yes Heaven forbid!
Posts
6 comments:
It is great to see the good old bell curve being wheeled out! Nassim Nicholas Taleb has written a lot on this topic. More interesting again is the fractal approach of Benoit Mandelbroit.
To suggest that Goldman Sachs are not compitent to mind our money is to miss the point completely.
The chances of this event happenning by random occurrence is so devistatingly remote, one can only conclude that the event was deliberately made to happen.
Thanks for another informative article I glade to read this.
Laby[man suit]
You are aware hopefully that the entire article assumes the true distribution is normal. With a fat-tailed distribution, which is a much better model for financial returns, you can have 25 sigma events occur much more frequently.
It's not so rare when you use a distribution with a better fit. Remember, all models are wrong, but some are useful. The normal distribution in this case is dangerous to use!
Even with an extremely fat-tailed distribution, the probability of getting multiple 25-sigma events in a row is very small. Chebyshev's inequality guarantees that the probability of a single 25-sigma event is ≤ 0.0016. Getting, say, three of these in a row has a probability of at most 4.096 × 10^-9. And this bound is almost certainly very poor (it is only sharp for distributions supported only on extreme points and the mean), so a more realistic probability should be many orders of magnitude smaller. This isn't impossible, but it's certainly not the most likely explanation.
I think it's pretty clear that the models were simply not accurate. They failed to predict these moves. And there could be other reasons on top of that, like simple exaggeration.
Post a Comment