This article appeared in The Malaysian Accountant journal, Jan-Feb 2011 issue Auditing Derivatives: Things that can

go wrong Value at Risk

By Jasvin Josen

Introduction Value at Risk or just VaR has been around for many years and is a major risk management task of any investment bank. VaR has not been receiving good press lately, especially when it was accused for not foreseeing the 2008 crisis. In this article, we will analyse what can go wrong with value at risk and discuss the auditors approach.

What is Value at Risk VaR is a technique for predicting the maximum losses from a portfolio of assests with a known level of confidence. For example, VaR might predict with a 95% confidence level that the maximum losses from a portfolio of risky assets would be RM 100,000. The effectiveness of this tool is in its ease of comprehension, making it a commonly used figure in boardrooms, with regulators and in annual reports. VaR was developed as a risk assessment tool in the 1990s, driven by the failure in the risk tracking systems used in the early 1990s to detect dangerous risk taken by traders. It became so widely accepted that regulators like the Basle Committee allowed banks to calculate their capital requirements for market risk with their own VaR models, using certain parameters provided by the committee. Trades in any portfolio may consist of stocks, bonds, commodities, currency trades or derivatives. Their mark-to-market values will change depending on how the market factors (equity prices, interet rates, currency rates...) change. VaR attempts to develop techniques to change these market factors in a robust manner, to observe the possible changes to the portfolio value. Three main basic methods are used historical simulation, variance-covariance and Monte Carlo simulation. Briefly, the historical simulation approach looks back at previous market factors behaviour and applies that same behaviour into the current portfolio to observe the changes in portfolio value.

in the Monte Carlo approach the market factors are simulated according to a defined statistical distribution. Then the simulated market factors are applied to the current portfolio to observe the changes in value. the variance-covariance approach works differently. It derives historical risk measures like standard deviation and correlation for each market factor. Then it applies these risk measures to the current portfolio, (after breaking down the portfolio into standard instruments) to derive the value at risk.

What can go wrong in using VaR? VaR cannot sense a catastrophic event as it pays attention to normal loss and not abnormal loss It is said that VaR is a measure of normal market risk. But what is normal? If one says that there is a 5% probability that a portfolio will not lose more than $100,000 in the following week, how sure are we that this is a normal loss? The choice of the confidence level is open to interpretation. Joe Nocera of New York Times (Jan 4, 2009) suggests that VaR was useful to risk experts but nevertheless exacerbated the crisis by giving false security to bank executives and regulators. The 2008 crisis dealt with new products like the Credit Default Swaps. These products generate small gains and very rarely have losses. But when they do have losses, they are huge. As such, it was outside the 99 percent probability and did not show up in the VaR number as it was lost in the tails. Some critics claim that VaR looks at only a small slice of the risk and a great deal of valuable information in the distribution is ignored. Manageable risk near the centre of the distribution is focussed on and the tails are ignored. What does the Auditor do? Depending on the level of importance that is placed on VaR in a firm, the auditor should observe if risk managers have a clear comprehension of all the important confidence levels and are able to place enough importance to them. Ignoring important risks -- contagion and liquidity Nassem Taleeb (the writer of The Black Swan) is concerned with what one calls measureable risks. Measurable risk is when you have a handle on the randomness. If I throw a pair of dice, for example, I can pretty much measure my risk because I know that I have one-sixth probability of having a three pop up. Non-measurable uncertainty is when

I'm throwing the dice without knowing what's on them. In the real world, most social events are non-measurable because nobody hard-coded the rules of the game VaR attempts to measure only market risk the risks that directly change the value of financial instruments in the markets. But there are other risks which are just as important contagion and liquidity. Although VaR models take into account the increased risk brought on by leverage, it fails to distinguish between leverage and liquidity risk borrowed money market instruments can be called in at any time and cause a major liquidity crisis. Liquidity crises can do serious damage to dynamic hedging to rebalance the portfolios a major panic that dries up liquidity will cause unreasonable bid-ask spreads, making hedging extremely difficult. The contagion effect which happens in no certain pattern in periods of market stress will break down correlations assumptions in VaR models, particularly the variance-covariance model. What does the Auditor do? The auditor must watch for the extent of leverage in the firm and market to deduce the appropriateness of VaR to interpret risk. He could discuss with management to assess the importance placed on contagion and liquidity risk which are not measureable but detrimental to the market and economy. Relying on the wrong past This is the problem of the historical simulation where the prior N days from which the historical sample was drawn upon is not representative of the present. One cannot be confident that errors of this sort will average out. Traders will know whether the actual prices changes over the last 100 days were typical, and therefore will know for which position the VaR is underestimated or overestimated. If VaR is used to set risk or position limits, the traders can exploit their knowledge of the biases in the VaR system and expose the company to more risk that the risk management committee intended, creating an incentive to take excessive but remote risks. What does the Auditor do? Alertness to industry and market knowledge will prompt the auditor to have considerable intuition of the appropriateness of the past data to represent the present. In times of disagreement, he should take it up with management. Wrong distribution assumption

The variance-covariance approach is the popular choice for investment banks to compute value at risk as it is easier to implement. However, a major assumption in the model is that returns of all assets are normally distributed. If the actual returns are not normally distributed, the computed value at risk will surely understate the true value at risk. The Monte Carlo approach also has a tendency to choose the Normal Distribution to derive the random market factors. But this method also allows the designer to choose another statistical distribution. However, this flexibility could lead him to making a bad choice. The chosen distribution might not adequately approximate the actual distribution of the market factors. The normal distribution assumption is far more detrimental for credit spreads. Credit spreads exhibit huge jumps and is proven to be nowhere near the Normal Distribution. For this, Credit VaR models are designed alongside the mainstream VaR models. What does the Auditor do? Backtesting procedures discussed later could give the auditor a good insight to the gaps between the actual and assumed distribution of asset returns. Input errors and measurement errors In the variance covariance approach, even if the returns distribution assumption holds up, the value at risk can still be wrong if the variance and covariance estimates are incorrect. There is because of a problem called the standard error. To quote Taleebs example: When I use a thermometer, I may be aware that there is one or two degrees of error in my measurement of the temperature. But here, I don't know much about the instrument, particularly when it comes to rare events A related problem occurs when the variance and covariance across assets change over time. Fundamentals that drive these numbers do change over time. For example, the correlation between the USD and MYR may change if oil prices increase by 20%. This can lead to a breakdown in the computed VaR number. What does the Auditor do? While recognising the inherent problem of the standard error, the auditor could review for scientific efforts to minimise this error. As for non-stationary of risk measures, the auditor could review for efforts to determine its impact. For example, one could break down the historical data into shorter periods (or by major events) to observe for any significant changes in the risk measures. Some VaR models are not designed for options

The limitation of the variance-covariance approach is that it incorporates options by replacing them with delta-equivalent spot positions. This is to linearise the options positions. However, options are not linear instruments. The price of the option does not behave in the same manner as the underlying price. This method can only work with moderate option positions, in a very short holding period. What does the Auditor do? The auditor needs to watch for the level of options and option-like products (e.g. convertible bonds) included in portfolios. Other considerations for the Auditor Other than the above concerns, the following considerations could prove useful for the auditor to determine the effectiveness of VaR. Improvements and Alternatives to VaR Over the years there has been a lot of effort to make value at risk a more reliable measure. Some deal with the non-normality of asset returns, some with the non-linearity problem for options, and others try to counter the non-stationary problem in volatility with time series models. There is also the inverse risk logic that does not attempt to calculate probabilities but adopts a stress testing approach, in a more realistic way to explain where the pressures are. [reference: Asian Link, January 2010: The Inverse Risk Logic Strategic Paradigm by Dr. David Bobker] Bearing in mind that VaR was built under the assumption of asset normality, probability of losses and value at risk are the simplest to compute with the normal distribution. It gets progressively more difficult with asymmetric and fat tailed distributions. In the course of the auditors work, especially in the present environment where instruments are getting more complex and plentiful in the Asian investment banks, the auditor should be attentive to these improvements and the extent it could complement the downside of VaR. Back test, Back test, Back test! Concerns about the reliability of any method can be addressed by comparing actual changes in value to the model numbers; and this applies to VaR as well. It is performed by comparing a sample of VaR numbers and the actual mark-to-market portfolio profit and losses, and answering two questions;

(i) (ii)

does the distribution of actual mark-to market profits and losses appear similar to the distribution used to determine the VaR amount? do the actual losses exceed the VaR amount with the expected frequency?

However, chances are that the distribution of the actual portfolio returns will almost always differ from the expected distributions. Because of this, very large samples may be required. Nevertheless, it provides the auditor with a good feel of the VaR numbers and how much reliance to place on it. Stress Testing Value at risk tells us that the loss will not exceed a certain amount but how much can the losses be? Stress testing usually acts as a complimentary tool. It is a set of scenario analyses to investigate the effects of extreme market conditions. However, parameters are stressed purely on the judgement and experience of the risk manager, for example 5 or 10 standard deviations away from the mean or by picking actual extreme events. The auditor must review stress testing results hand in hand to compliment the VaR and watch for the adequacy of scenarios. The human element Focussing on quantitative techniques will cause one to miss major problems that occurs systemically like over-leveraged firms, rogue trading and fraud. The auditor must always be mindful of human factors that are usually the main cause of downfalls. Another impact of the human element is in the VaR itself. The effectiveness of VaR depends of the people who understand the technology, interpret the results of VAR analysis, balance it with other means of testing and articulate the VaR results to management, shareholders and analysts. Conclusion Taleeb postulates that value at risk should be nothing but a small footnote in the way we view the risks, not the dominating tool. Damodaran (New York University) is of the opinion that it is more prudent to use all of the information in the probability distribution rather than a small slice of it. Mr Pasquier, head of investment risk at Axa Investment Managers says: It is necessary to have models but also to understand their limitations common sense and discussion are still very important.[source: Financial Times, Oct 17, 2010] The risk management objective function is survival, not profits and losses. To a risk manager, VaR is the level of losses at which you stop trying to guess what will happen next, and start preparing for anything that may happen. It is far more important to worry about

what happens when losses exceed VaR. It is perhaps best for the auditor to think the same way too.