Backtesting value at risk models in the presence of structural breaks on the Romanian and Hungarian stock

802 BACKTESTING VALUE AT RISK MODELS IN THE PRESENCE OF STRUCTURAL BREAK ON THE ROMANIAN AND HUNGARIAN STOCK MARKETS Zapodeanu Daniela, Kulcsar Edina, Cociuba Mihail Ioan University of Oradea, Faculty of Economics, Department of Finance, Oradea, Romania [email protected] [email protected] [email protected] Abstract: Transactions on financial markets are associated with variability, risk and uncertainty, so quantification of risk has a great importance. Beside Standard Deviation and Variance, one of the most involved risk measure methods is Value-at-Risk (VaR). In this study, we use daily return for the stock index from Romania (BET) and Hungary (BUX) for the 01:2007 - 02:2013 periods in order to test the influence of structural breaks on the VaR metrics. We find out that the ARCH phenomenon is present, so we use the GARCH family models. The structural breaks in the series mean and variance are identified using the Zivot-Andrews test and PELT algorithm, the structural break dates are captured using dummy variables in the GARCH models (struc-GARCH), the selection of models is done using the informational criterion [Akaike, Schwarz, Log-likelihood]. The results of present research show a greater volatility associated with a higher risk level in case of Romanian stock index. The stock market indices return follows a negatively skewed and leptokurtic distributions forms either in two cases, so is unspecific a normal distribution. After applying above mentioned tests we can conclude that there are eight structural breaks in BET index returns variance and there are five breakpoints in case of BUX. The breakpoints in mean show very closely results in time, for BET in February 2009 and for BUX March 2009. Backtesting VaR models are done by measuring the number of times the loss is greater than the VaR forecast. The first step for unconditional coverage testing consists in comparing of fraction of VaR violation for a particular risk model. The independence testing it is very important tool in back-testing, because it is not the same that the VaR violations are differentiated in time or there are clustered in some certain period. By checking the independence test, we have the possibility to discover and reject the model with clustered hit sequence. Testing the influence of structural breaks on VaR we find that incorporating structural breaks in the GJR-GARCH models generates lower violations when comparing with the plain GJR-GARCH or RiskMetrics methodology. Keywords: stock market, structural break, Value at Risk, GARCH JEL classification: G17 1. Introduction Nowadays, economic and financial environment most dominant characteristics are instability, variability, risk and uncertainty, but a well known economic principle says: “no risk means no gains”. When we deal with risk, uncertainty and volatility, first it is essential to treat the main difference between these three concepts. According to Keynesian approach, there isn’t a significant difference between first two concepts, Knight considers the contrary that there is a sharp distinction between risk and uncertainty in his famous work “Risk, Uncertainty and Profit” (1921). He considers that the most important difference between risk and uncertainty consists in the possibility of quantifying, so in case of risk we could make measurements but in the case of uncertainty we can’t. What about variability? In some cases, the concept of variability is considered to be the main 803 component of risk besides uncertainty (Molak, 1997; Cullen–Frey, 1999). Others consider the essence of this represent the temporal and spatial heterogeneity of values (Molak, 1997). Risk cannot be completely avoided for the financial markets participants, but there are many ways for managing and minimizing it. This paper aims to present the principal risk categories which are specific to financial markets products, how they affect the stock market participant’s behavior and also the choosing of risk management alternatives. The main objective of this paper consists in quantification of risk with VaR method for two neighboring countries main stock index returns: BET for Romania and BUX for Hungary, and testing the influence of structural breaks in mean and variance on the VaR metrics. The remaining of the article is organized as follows: in section 2 we review the literature with the main steps of risk management and the most used risk quantification methods, concentrating on Value-at-Risk method and on his advantages and limits. In Section 3 we present the research methodology, consisting in mathematical background of Value-at-Risk, the main characteristics of GARCH models and used tests for validation of structural break points. The data analysis part contains the evolution of returns in the analyzed period and the main statistics of it, we also presented in this part the results of stationary test and the dates of structural breaks in means and variance for studied stock market index. The next part shows the results of study, exactly the equation of mean. The last part of study show the main conclusions after analyzing these two stock market index risk. 2. Literature review According to Horcher (2005) the risk management is a very broad concept, which includes more steps. First and the most important step represent the identification and quantification of the internal and external risk factors, and the specific risk categories which could affect expected gains and returns; the second is ranking of risks by priority and possible losses; next step define a risk tolerance level, which can be supported; the last one and also the most consistent step is developing the risk management strategies, which includes also risk minimizing methods. In the risk management process, we try to concentrate in this paper to the first step, on risk and risk factor identification process and especially to risk quantification methods. One of the most used methods in financial markets risk exploration is Value at Risk (VaR), which was developed in the ’90 years by J.P. Morgan. In this period the method was used with great success by central banks, and after that becomes more popular among financial institutions (Chen, 2007). In our days, this method it’s also used at company level in financial risk quantification like market risk, credit risk, liquidity risk etc. The VaR method is often used to estimate the level of exchange rate risk, but is also suitable for portfolio risk measurement. Based on statistical probability estimations, the essence of VaR method consist in quantification of maximum potential loss, which result from market factors variability. Therefore, the VaR determine the level of maximum expected loss, for different time periods from 1 day to 100 days, at specific confidence level 95%, 97, 5%, or 99%. The VaR is the only one method which characterized the level of portfolio, investment risk by a number. So, one of a great advantage is that characterized risk with a number. Another big advantage is that could be well completed by other risk measurement methods, such as scenario analysis and stress testing and sensitivity analysis methods. Manganelli & Engle, (2001) classify the Value-at-risk models in three categories: parametric (RiskMetrics, Garch), nonparametric (historical simulation, hybrid model), semi-parametric (Extreme Value Theory, CAViaR, quasi-maximum likelihood Garch). In practice, the application of VaR knows three methods: first based on historical data, second the method of variance and covariance or parametric based 804 method, and the third based on Monte Carlo simulation (Horcher, 2005). The advantage of the first method is that permits quick and easy usage, but the last two methods provide much more accurate results and have a wider range of applications. The VaR calculation based on historical data assumes that past data and events also characterized the future events. The VaR estimation based on Monte Carlo simulation is the most flexible method, which basically consists in random number generator, which is often used in financial modeling. The success of this method is determined by the success of used valuation method, but also depends on the used parameters in the simulation (Ray, 2010). The major disadvantage of VaR risk measurement method is that it couldn’t be applied in the extreme, shock conjuncture, such as financial crisis. The abrupt and significant fluctuations of risk factors greatly deform the efficiency VaR method. To eliminate this problem, Artzner et al. (1997, 1999) developed the Expected Shortfall (ES) concept, which characterized the conditional expected loss which exceeds the value of loss received by using of VaR method (Yamai et al, 2002). The ES method is closely related to the VaR, because we could obtain the Expected Shortfall value from VaR value by attaching probability levels to expected loss. The great advantage of ES method is that take into account the possibility of extreme situations (Kerkhof, 2003). Artzner et al. (1999) considers Expected Shortfall (ES) method a more coherent risk measure method than Value at Risk (VaR). Cuoco, He, Issaenko (2001) in their research, conclude that the multiple uses of VaR and ES methods generate equally results. 2. Methodology The Value-at-risk is defined by (McNeil et al, 2005) as at “... some confidence level the VaR of the portfolio at the confidence level is given by the smallest number l such that the probability that the loss L exceeds l is not larger than “. Mathematically we can write VaR as probability: (1) Engle (1982) in his seminal paper proposed the autoregressive conditional heteroskedasticity models which view the variance as being dependent of the errors, the ARCH model was extended in the Generalized Autoregressive Conditional Heteroskedasticity (GARCH) by Bollerslev [1986] which has the following form: (2) (3) (4) Because GARCH model treats the shocks symmetrically while on the financial markets bad news generates more volatility than the good news. Glosten, Jagannathan and Runkle [1993] proposed Threshold GARCH which treats differently the bad-good news influence on the assets prices. It is an asymmetric model in which the conditional volatility is: (5) where: dt = 1 if et<0 or dt = 0 if et> 0. The detection of breakpoints in time series can be posed (Killick, Fearnhead, & Eckley, 2012) as a hypothesis test where H0 is the null hypothesis where there is no changepoint (m=0) and the alternative hypothesis H1 where we have at least 1 changepoint (m>=1). Killick et al. (2012) developed the Pruned Exact Linear Time (PELT) method which test for changepoints using the following statistical criteria: penalized likelihood, quasi-likelihood and CUSUM; the breakpoint analysis is carried in the mean, variance and both mean/variance of the series. The PELT method is implemented as an R packaged (changepoint package). The test statistics used in the PELT method implementation (Killick et al., 2012) has the following null hypothesis H0: no breakpoint and alternative hypothesis H1: one breakpoint τ1, with τ1 {1, 2,…, n-1}. By rejecting the null 805 hypothesis H0, a changepoint is detected and it is estimated by maximizing the loglikelihood. We apply the following unit-root test: Augmented Dickey–Fuller test (ADF) and Phillips–Perron (PP) and the Zivot and Andrews (1992) which extended the Dickey–Fuller test by allowing for a break in intercept, trend and both (model C). The structural break will be introduce in the GARCH model equations using a dummy variable, also in order to eliminate autocorrelation lags of dependent variable will be introduce in the main equation, the model will be as follows: (6) (7) where Dmi ,..., Dhi are dummy variables which take the value 0 before the breakpoint and 1 after the breakpoint until the end of the period. Backtesting VaR models is done by measuring the number of times the loss is greater than the VaR forecast, the number of VaR violations can be define as: (8) For an improved risk model it is necessary to predict the probability of VaR violations, noted with p. The VaR violation probability depends on the coverage rate of VaR, the hit sequence from a correctly specified risk model looks like a sequence of random tosses of coin (Christoffersen, 2012). The first step for unconditional coverage testing consists in comparing of fraction of VaR violation for a particular risk model. The independence testing it is very important tool in back-testing, because it is not the same that the VaR violations are differentiated in time or there are clustered in some certain period. By checking the independence test, we have the possibility to discover and reject the model with clustered hit sequence. The first step consist in assuming that violations are dependent over time, which could be described better by Markov transition probability matrix. The Markov property refers to the assumption that only today’s outcome is determinant for tomorrow outcome, the evolution from the past doesn’t matter, where π11 is the probability of tomorrow being a violation given today is also a violation and π01 is the probability of tomorrow being a violation given today is not a violation. For checking the independence π01 = π11, a likelihood ratio test is used. After using the independence test, the next step for correct coverage is the conditional coverage test, which checks that π01 = π11 = p. The test is computed by summing of unconditional coverage and independence test. 3. Data analysis The analyzed series are two stock exchange index: BET for Romania and BUX for Hungary, the analyzed period is between 01:2007 - 03:2013, daily series; the data are obtained from www.bvb.ro and www.bet.hu ; the econometrics software used are Gretl and R package strucchange, in order to obtain returns from the daily series we apply the following transformation: where i= BET, BUX. 806 Fig. 1 The evolution of r_BET and r_BUX Table 1. Descriptive statistics Series Mean SD SK KT Q(12) Q2(12) JB r_BET -0.0259 2.00 -1.19 17.40 48 290 13605 r_BUX -0.0212 1.88 -0.03 9.27 62 1088 2515 Notes: SD, SK, KT, and JB denote standard deviation, skewness, kurtosis, and Jarque-Bera statistic, respectively. The Ljung–Box statistics, Q and Q2 stat checks for serial correlation of returns and squared returns up to the 12 order, the critical value for the Q(12) respectively are 26.21, at 1% significance level. The critical value for the Jarque-Bera (JB) test is 5.991 at 5% significance level. Table 1 presents the descriptive statistics of the returns on daily series of closing prices of two stock exchange indices: BET for Romania and BUX for Hungary. For the analyzed period, 01:2007 - 03:2013 both returns are negative, the lowest value is observed in case of Hungarian stock index, -0.0212; the standard deviation, reveals a higher volatility in the case of Romanian stock index, BET. The stock market indices distributions are negatively skewed, the kurtosis value is higher than the normal distribution kurtosis value, which is 3, so we could see for the both analyzed series, that the series have leptokurtic distributions. The Jarque-Bera test indicates that the normality of distribution for r_BET and r_BUX is rejected, also the Q-statistics indicates serial correlation of returns which will be removed using lag terms and the serial correlation of squared returns Q2 suggests the existence of the ARCH effect. Testing for the ARCH effect is done using the LM test, the LM value for the r_BET, r_BUX are 148.19, respectively 381.40, which compared with the critical values shows the presence of ARCH(1) effects, so the series will be modeled using the GARCH family models. Table 2. Unit Root/stationarity test Series ADF PP KPSS r_BET -9.0041 -37.419 0.27484 r_BUX -16.197 -36.523 0.088335 MacKinnon’s 1% critical value is -3.46 for the ADF and PP tests, the critical value for the KPSS test is 0.739 at 1% significance level, * denote significance at 1% levels. 807 Table 2 present the result for the unit root test, where ADF and PP test has the null hypothesis that the series is integrated of order 1, while KPSS null hypothesis is that the series is stationary, and Zivot-Andrews test allows for a break in intercept, trend or both. We find that for r_BET and r_BUX we reject the unit-root hypothesis based on the ADF test, PP test. We find that the returns don’t have a unit-root, the KPSS accept the stationarity of returns for both countries. Based on the results from the ADF, PP and KPSS tests we apply the Zivot-Andrews test and breakpoint analysis in order to capture any break in mean, variance or both. Table 3. Breakpoint dates Series Mean breakpoint Zivot-Andrews Variance breakpoint r_BET 2009-02-13 2009-02-24*** (-11.01) 2007-12-19, 2008-09-11, 200810 -27, 2009-08-19, 2010-0430, 2010 -07-01, 2011-0729, 201110-28 r_BUX 2009-03-11 2009-03-11*** ( -12.51) 2008-09-12, 2008-11-21, 201006 -03, 2011-07-28, 2012-0119 Test statistics in parenthesis, the critical value for Zivot-Andrews test is -5.34 at 1% significance level. Table 3. and Figure 2. presents the estimated breakpoints, and in order to capture the changes in mean and variance dummy variables will be introduce in the GARCH model equations. We find for the r_BET series there is two breaks in the mean equation both of them in February 2009 and eight breaks in the variance: 2007-12-19, 2008-09-1, 200810-27, 2009-08-19, 2010-04-30, 2010-07-01, 2011-07-29, 2011-10-28; for the r_BUX series there is one break in the mean equation in March 2009 and five breaks in the variance: 2008-09-12, 2008-11-21, 2010-06-03, 2011-07-28, 2012-01-19. 808 Fig. 2. Variance breakpoints in BET, BUX 4. Results The GARCH model will incorporate the non-normality of BET and BUX return, the Jarque-Bera test indicates that the series follow a non-normal distribution, by using the normal and student distribution for the errors, also threshold models will be tested due to the asymmetries of financial series, the r_BET and r_BUX series are skewed to the left (negatively skewed); in order to eliminate the serial correlation autoregressive lags will be used in the mean equation. Table 4. GARCH models r_BET r_BUX constant no no AR lags 1**, 13** 2**, 4** Mean breakpoint (1) no no Zivot-Andrews no no Variance breakpoint 2007-12-19**, 2008-09-11*, 200908-19**, 2010-04-30*, 20100701**, 2011 -07-29**, 2011-1028** 2008-09-12** 2010-06-03** GARCH (p,q) (1,1)*** GJR (1,1)*** GJR Q(6) 3.80 7.01 Q(12) 11.21 10.52 Q2(6) 6.05 2.77 Q2(12) 14.00 5.25 Where p is the number of lagged h terms and q the number of e2 terms. *, **, *** denote significance at 10%, 5%, and 1% levels. 809 The GARCH models are selected based on their information criterion, not presented here, and the stability of the models. Based on the results of GARCH models estimation, we can observe in Table 4., with bold, the significant breakpoints, we find no breaks in the intercept and we find 7 structural breaks in the variance term in the case of Romanian stock market and 2 structural breaks in the variance term for Hungarian stock market. Backtesting the Value at Risk is done on the RiskMetrics model, the vanilla GJR(1,1) and the variance break GJR model, the GARCH in mean model is not representative for the analyzed series. Table 5. Number of hits BET RiskMet rics 1% RiskMet rics 5% GJR(1,1 ) 1% GJR(1,1 ) 5% GJR(1,1 ) 1% with breaks GJR(1,1) 5% with breaks hits 2 14 2 9 1 6 consecutive hits 0 2 0 0 0 0 β 0.13% 0.92% 0.13% 0.59% 0.07% 0.40% BUX RiskMet rics 1% RiskMet rics 5% GJR(1,1 ) 1% GJR(1,1 ) 5% GJR(1,1 ) 1% with breaks GJR(1,1) 5% with breaks hits 1 5 0 3 0 1 consecutive hits 0 0 0 0 0 0 β 0.07% 0.33% 0.00% 0.20% 0.00% 0.07% 5. Conclusion We can observe from the analyzed times series that neither BET or BUX indices don’t follow a normal distribution, in both cases we can see negatively skewed and leptokurtic distributions of daily returns. Analyzing standard deviation we can conclude that a greater risk and volatility is specific for Romanian stock market index return. Because the normality and LB tests reveal that the ARCH effect is present in case of BET and BUX we applied GARCH models. After we applied the breakpoints analysis we find out that structural breakpoints are present in both cases. For BET index we observe eight structural breaks in variance, while in the case of BUX only five breakpoints are present. The Zivot-Andrews and PELT method shows that the mean breakpoints are very close for the two markets, February 2009 for BET index returns and March 2009 for BUX index 810 returns, which could be related with the consequences of economic and financial crisis which debuted in Central and Eastern Europe region in October 2008. Testing the influence of structural breaks on VaR we find that incorporating structural breaks in the GJR-GARCH models generates lower violations when comparing with the plain GJR-GARCH or RiskMetrics methodology. References ■ Artzner, P., Delbaen, F., Eber, J.-M., Heath, D. (1997) Thinking coherently. RISK 10(11). ■ Artzner, P., Delbaen, F., Eber, J.-M., Heath, D. (1999) Coherent measures of risk. Math. Fin. 9(3), 203–228. ■ Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of econometrics, 31(3), 307–327. Elsevier. ■ Chen, F. - M.Sc. Ying (2007): Adaptive Risk Management, Dissertation, 27 p. ■ Christoffersen, Peter F. Elements of financial risk management. Academic Press, 2012. ■ Cullen, A.C., Frey C.H. (1999): Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing Variability and Uncertainty in Models and Inputs, Plenum Press, New York, 335 p ■ Cuoco, D., H. He, and S. Issaenko (2001), Optimal Dynamic Trading Strategies with Risk Limits, Manuscript, Yale University. ■ Engle, R. F. (1982). Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation. Econometrica, 50(4), 987. doi:10.2307/1912773 ■ Glosten, Lawrence R., Ravi Jagannathan, and David E. Runkle. "On the relation between the expected value and the volatility of the nominal excess return on stocks." The journal of finance 48.5 (1993): 1779-1801. ■ Horcher, K. A. (2005): Essentials of financial risk management, John Wiley & Sons, Inc., Hoboken, New Jersey., 217 p. ■ Kerkhof, F., Lambertus, J. (2003): Model Risk Analysis for Risk Management and Option Pricing, doktori értekezés, Tilburg ■ Killick, Rebecca, Paul Fearnhead, and I. A. Eckley. "Optimal detection of changepoints with a linear computational cost." Journal of the American Statistical Association 107.500 (2012): 1590-1598. ■ Knight, F. H. (1921): Risk, Uncertainty and Profit. Reprint London School of Economics, 1933 ■ Manganelli, S., & Engle, R. F. (2001). Value at risk models in finance. ECB Working Paper No. 75. ■ McNeil, A.; Frey, R.; Embrechts, P. (2005). Quantitative Risk Management: Concepts Techniques and Tools. Princeton University Press ■ Molak, V. (1997): Fundamentals of risk analysis and risk management. Lewis Publishers (CRC Press, Inc.), New York, 457 p. ■ Ray, C. (2010): Extreme risk management. Revolutionary approaches to evaluating and measuring risk, McGraw-Hill Companies, 37 p. ■ Yamai, Y. – Yoshiba, T. (2002): Comparative analyses of expected shortfall and value-atrisk under market stress, IMES Discussion Paper No 2002-E-2, Bank of Japan ■ Zivot E., DWK Andrews; Further evidence on the great crash, the oil-price shock, and the unit-root hypothesis. Journal of Business and Economic Statistics, 10 (1992), pp. 251– 270.