Backtesting Value-at-Risk Models

1 IntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting Value-at-Risk ModelsChristophe HurlinUniversity of Orl ansS minaire Validation des Mod les Financiers. 29 Avril 2013 Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsIntroductionThe Value-at-Risk (VaR) and more generally the Distortion RiskMeasures (Expected Shortfall, etc.) are standard risk measuresused in the current regulations introduced in Finance (Basel 2), orInsurance (Solvency 2) to x the required capital (Pillar 1), or tomonitor the risk by means of internal risk Models (Pillar 2).

2 Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsIntroductionDe nitionLetfrtgTt=1be a given P&L series. The daily (conditional) VaR fora nominal coverage rate is de ned asPr[rt< VaRtjt 1( ) Ft 1]= whereFt 1denotes the set of information available at timet HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsIntroductionWho does use VaR? What for?Bank risk managerMeasure rm-level market, credit, op. riskBank executivesSet limits (management)Banking regulatorsDetermine capital requirementsExchangesCompute marginsRegulatorsForecast systemic risk (CoVaR)IndustryEx.

3 EDF, spot prices of electricityChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandations"Disclosure of quantitative measures of market risk, suchas Value-at-Risk , is enligthening only when accompaniedby a thorough discussion of how the risk measures werecalculated and how they related to actual performance",Alan Greenspan (1996)Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsIntroductionDe nitionBacktesting is a set of statistical procedures designed to check ifthe real losses are in line with VaR forecasts (Jorion, 2007).

4 Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsIntroductionWha tever the type of use of VaR, the VaR forecasts aregenerated by aninternal risk model is used to produced a sequence of pseudo out-ofsample VaR forecasts for a past period (typically one year)The Backtesting is based on the comparison of the observedP&L to these VaR HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsOutlines1 How to test the validity of a VaR model ?

5 2 What are the Backtesting strategies?3 What are the good practices?Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesRemark 1:Ex-post VaR is not observable, so it is impossible tocompute traditional statistics or criteria such as 2:There is no proxy for the VaR contrary to the volatility(realized volatility, Andersen and Bollerslev 1998)Patton, (2011) Volatility forecast comparison using imperfect volatilityproxies, Journal of Econometrics, 260, HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesBacktesting procedures are based onVaR exceptionsDe nitionWe denoteIt( )the hit variable associated to theex-postobservation of an %VaR exception at timet.

6 It( )=(1 ifrt< VaRtjt 1( )0 elseChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesChristo ersen (1998) : VaR forecasts are valid if and only if theviolation processIt( )satis es the following two assumptions:1 Theunconditional coverage(UC) (IND) ersen (1998), Evaluating interval forecasts, International EconomicReview, 39, pp. HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesDe nition (unconditional coverage hypothesis)The unconditionnal probability of a violation must be equal to the coverage ratePr[It( )=1]=E[It( )]=.)

7 If Pr[It( )=1]> ,the risk is under-estimatedIf Pr[It( )=1]< ,the risk is over-estimatedChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesDe nition (independence hypothesis)VaR violations observed at two di erent dates must beindependently ( )andIs( )are independently distributed fort6=sChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesFigure:Illustration: violations cluster050100150200250-6-4-202468 VaR(95%) P&LChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesFigure:Illustration.

8 Violations cluster050100150200250-6-4-202468 VaR(95%) P&LChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesDe nition (conditional coverage hypothesis)The violation process satis es a di erence martingale [It( )jFt 1]= Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsBacktesting PrinciplesRemark:These assumptions can be expressed as the UC assumption, each variableIt( )has a Bernouillidistribution with a probability.

9 Itt( ) Bernouilli( )Under the IND assumption, these variables are independent, andthe number of violations has a Binomial t=1It( ) B(T, )Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsFrequency-based testsMagnitude-based testsMultivariate testsIndependence testsDuration-based testsTesting strategiesWhat are the Backtesting strategies?Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsFrequency-based testsMagnitude-based testsMultivariate testsIndependence testsDuration-based testsTesting strategiesLet us consider a sequence of daily VaR out-of-sample forecasts VaRtjt 1( ) Tt=1and the corresponding observed P& to test the validity of the internal risk model ?

10 Hurlin C. and P rignon C. (2012), Margin Backtesting ,Review of Futures Market, 20, pp. 179-194 Christophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsFrequency-based testsMagnitude-based testsMultivariate testsIndependence testsDuration-based testsTesting strategiesTesting strategies:1 Frequency-based tests2 Magnitude-based tests3 Multivariate tests4 Independence tests5 Duration-based testsChristophe HurlinBacktestingIntroductionBacktesting PrinciplesTesting strategiesRecommandationsFrequency-based testsMagnitude-based testsMultivariate testsIndependence testsDuration-based testsTesting strategies: frequency-based tests (1/5)Figure:BIS "Tra c Light" SystemNote.