An improved model accuracy for forecasting risk measures: Application of ensemble methods

Makatjane, Katleho; Mmelesi, Kesaobaka Article An improved model accuracy for forecasting risk measures: Application of ensemble methods Journal of Applied Economics Provided in Cooperation with: University of CEMA, Buenos Aires Suggested Citation: Makatjane, Katleho; Mmelesi, Kesaobaka (2024) : An improved model accuracy for forecasting risk measures: Application of ensemble methods, Journal of Applied Economics, ISSN 1667-6726, Taylor & Francis, Abingdon, Vol. 27, Iss. 1, pp. 1-30, https://doi.org/10.1080/15140326.2024.2395775 This Version is available at: https://hdl.handle.net/10419/314291 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Journal of Applied Economics ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/recs20 An improved model accuracy for forecasting risk measures: application of ensemble methods Katleho Makatjane & Kesaobaka Mmelesi To cite this article: Katleho Makatjane & Kesaobaka Mmelesi (2024) An improved model accuracy for forecasting risk measures: application of ensemble methods, Journal of Applied Economics, 27:1, 2395775, DOI: 10.1080/15140326.2024.2395775 To link to this article: https://doi.org/10.1080/15140326.2024.2395775 © 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. Published online: 08 Sep 2024. Submit your article to this journal Article views: 537 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=recs20 REVIEW ARTICLE An improved model accuracy for forecasting risk measures: application of ensemble methods Katleho Makatjane a and Kesaobaka Mmelesi b a Department of Statistics, University of Botswana, Gaborone, Botswana; b School of Economics and Econometrics, University of Johannesburg, Johannesburg, South Africa ABSTRACT Statistical-based predictions with extreme value theory improve the performance of the risk model not by choosing the model structure that is expected to predict the best but by developing a model whose results are a combination of models with different shapes. Using different ensemble algorithms to conglomerate the TBATS and the GEV distribution, we found that the stacking ensemble algorithm outperforms other ensembles hence the forecasting accuracy of risk measures is improved with the stacking ensemble algorithm. The risk estimates suggest that the returns on losses averaging 0.014 and 0.018 invested at 90 and 99 percent respectively, are riskier than the returns on gains. Backtesting results further revealed that all the risk measures are reliable, and the combined model is a good one for computing financial risk particularly in South Africa. At high confidence levels, all the risk measures seem to perform better than at lower confidence levels, as evidenced by higher probability values from backtesting using the Kupiec and Christoffersen test at 95 percent than at 99 percent levels of significance. ARTICLE HISTORY Received 18 January 2024 Accepted 14 August 2024 KEYWORDS combined forecasting; financial markets; extreme value theory; machine learning JEL CLASSIFICATION C13; C22; C52; C58 1. Introduction From the viewpoint of financial risk managers, a risk measure can be viewed as a map from the space of probability distributions to actual numbers. Risk management professionals can adjust their capital reserves against the downside risk by using risk measures to give banks and other financial institutions specific values for potential losses. Two prevalent measures of financial risk that rule modern financial regulation are value-atrisk (VaR) and expected shortfall (ES). In the worst-case scenario, at a specific confidence level, VaR gives banks and investment institutions a loss level (Lazar & Xue, 2020). Value-at-risk is not a cogent risk measure in the traditional sense and has inherent flaws because it ignores the shape and structure of the tail. As a result, following the financial crisis of 2007–2008, the BCBS (2014) herein referenced (Basel Committee on Banking Supervision) suggested switching from VaR, which had a confidence level of 99 percent, CONTACT Katleho Makatjane [email protected] Department of Statistics, University of Botswana, Gaborone, Botswana JOURNAL OF APPLIED ECONOMICS 2024, VOL. 27, NO. 1, 2395775 https://doi.org/10.1080/15140326.2024.2395775 © 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/ licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent. to ES, which has a confidence level of 97.5 percent. Given that its realisation must be lower than that of VaR, ES is the expected return on investment. On the other hand, deriving a model that investors and/or stock markets will use to track, predict, and forecast their daily gains or losses in real time is therefore crucial. As a result, forecasting risk is a huge and active research area that has piqued the interest of many different academic disciplines, including finance, engineering, and statistics among others. Due to this, a significant amount of literature has focused on techniques that can generate reliable forecasts in a range of real-world scenarios (Hajirahimi & Khashei, 2019). The literature typically points to two main approaches: (1) developing and proposing new forecasting models, and increasing the accuracy of obtained results. (2) Combining different forecasting model types. This is because no single comprehensive model can simultaneously capture all of the patterns present in the data; therefore, hybridisation is usually used. Therefore, the purpose of this study is to combine the TBATS with the generalised extreme value (GEV) distribution and compare the effectiveness of different ensemble methods in combining the two models. In this way, we aim to improve and, enhance the forecasting accuracy of risk measures, namely, Value-at-risk, expected shortfall, conditional tail expectation (CTE), and glue value-at-risk (Glue-VaR). Generally speaking, several models are estimated, and the most accurate model is chosen. Because of a few possible impacting factors such as sampling variance, model uncertainty, and structural changes, the final model chosen may not be the best for use in the future. With minimal additional work, the model selection problem is made easier by integrating various models. Secondly, time series data in the real world are rarely perfectly linear or nonlinear; because both linear and non-linear patterns are frequently present. In this scenario, neither the TBATS nor the GEV distribution can be used effectively to model and forecast time series data specifically the stock market time series, since the TBATS is not designed to handle the tail behaviour of the distribution and extreme values. While the GEV distribution on the other hand, cannot handle both linear and non-linear patterns equally well on its own. For that reason, we find it interesting to us to accurately represent complex time series structures by merging the TBATS with the GEV distribution and see how the new combined model performs on a five day business financial time series exchange/Johannesburg stock exchange (FTSE/JSE) all-share index. Thirdly, the forecasting literature virtually unanimously agrees that there is no single ideal strategy for every situation. This is mostly because real-world issues are frequently complicated, making it possible for distinct patterns to be captured differently by different models. Utilising the TBATS helps us to handle multiple seasonalities and irregular trends, making it suitable for us to capture the underlying dynamics of the data, while the generalised extreme value distribution on the other hand helps us to accurately capture extreme values in the tails of the distribution. The integration of this two models helps us to obtain more accurate estimates of tail risk, particularly in scenarios where extreme events are infrequent but significant; leveraging the strengths of both models to improve forecasting performance, particularly in situations where capturing extreme events accurately is crucial for risk management or decision-making. With this proposed three-stage hybrid model, in the fisrt stage we amalgamate the TBATS to obtain independently and identically distributed (i.i.d) residuals while at the same time modelling non-constant seasonal patterns over time. Instead, the model helps 2K. MAKATJANE AND K. MMELESI us to adapt to changes in seasonality, making it suitable for time series data with irregular or evolving seasonal patterns. On the second stage, we extract the block maximas for the positive returns (i.e., the gains) and block minimas for negative returns (i.e., the losses) from the i.i.d residuals of the TBATS model and we fit the GEV distribution to model volatility, extreme tail losses, and gains of the South African stock market. In detail, we consider a generalised extreme value distribution with a specification that the extreme value sequence comes from the autoregressive and the moving average (ARMA) errors that capture temporal dependencies and autocorrelation in the time series. The dependence is captured by an appropriate temporal trend in the local and scale parameters of the GEV distribution. The TBATS-GEVD modelling approach is believed to perform better in forecasting the risk measures and it is suited to explain extremes better than the classical methods; where the classical methods such as Autoregressive integrated moving average (ARIMA) among others cannot capture the tail behaviour adequately because they neither assume a normally distributed nor even fatter tailed distributed (e.g., t) innovations as suggested by Calabrese and Giudici (2015). This study is the first empirical analysis that employs TBATS in conjunction with the GEV distribution to quantify the likelihood of future extreme daily losses and gains. Extreme Value Theory (EVT) is one of the most useful techniques to predict extreme cases, which have a big effect but less occurrence probability. In this study, the Block Maxima and Minima (BM) approach, which is one of the main extreme value techniques, has been used. In this approach, there is no proper block size selection methodology (Özari et al., 2019). It has been observed that the block size selection for the estimation made in previous studies has been used randomly without relying on any assumption. It is necessary for new applications to find a certain block size that can be used for all of the data sets. Hence, the second main contribution of this study is the novelty of the application on how the optimal number of blocks to be fitted to the GEV distribution are selected. We propose a methodology to select the best block size for both maximal and minimal returns with a case study on the South African market all-share index which makes up the top 99 percent of the total pre-free-float market capitalisation of all listed companies on the Johannesburg Stock Exchange. The rest of this study is organised as follows section 1.1 represents literature review. Section 2 is the methodology followed. While section 3 presents result and discussion from data analysis and finally section 4 presents the conclusion of the study. 1.1. Literature review Conventional measures of risk in earnings based on historical standard deviation require long time-series data and are inadequate when the distribution of earnings deviates from normality. A methodology that is based on current fundamentals, and quantile regression to forecast risk reflected in the shape of the distribution for future earnings, was developed by Konstantinidi and Pope (2016). Even though VaR is widely used, many studies have thoroughly examined its drawbacks. When a loss distribution has fat tails, VaR is not only incoherent but also fails to accurately estimate the risk of the loss (Rockafellar & Uryasev, 2002), and therefore, this significantly undermines the reliability of this risk measure (Chen, 2018). Despite this, value-at-risk is still a popular risk indicator because it is so easy to compute and comprehend. In addition to demonstrating JOURNAL OF APPLIED ECONOMICS 3 the incoherence of VaR, Artzner et al. (1999) also introduced the expected shortfall and dubbed it the ideal risk indicator. Using coherent risk measure theory, Pflug (2000) further demonstrated that ES is a coherent risk measure. And, in contrast to VaR, which is more frequently used, expected shortfall is a risk measure that is sensitive to the shape of the tail of the distribution of returns on a portfolio. To correctly estimate risk measures, a distribution that captures all extreme and stylised facts is required. Extreme or “tail” risk, as described in BCBS (2019), requires risk practitioners to fully understand it. It has been developed to use the extreme value theory where Fisher and Tippett (1928) and Pickands (1975) are credited for developing the field of EVT. By obtaining three asymptotic limits, Fisher and Tippett (1928) were able to describe the distribution of extreme values, assuming that the variables are independent. An argument for modelling extreme values using a generalised extreme value distribution can be attributed to Maritz and Munro (1967). Extreme value theory is a theory that measures and models extreme events (large fluctuations) (tails of statistical distributions), i.e., it is suitable for financial assets with extreme returns (very large fluctuations in returns); hence, it assumes independent and identically distributed observations. This assumption of i.i.d does not necessarily hold for financial time series data (Chen, 2023). To correct this, McNeil et al. (2015) proposed a two-stage methodology in the form of a generalised autoregressive conditional heteroscedasticity (GARCH)-EVT model using five index returns in their illustrations. The first step was to capture the heteroscedasticity (non-constant variation or fluctuations) features by fitting the GARCH model. The second step was to apply the Generalised Pareto distribution (GPD) and the GEV distribution to the residuals extracted from a selected GARCH model. The advantages of the GARCH-EVT hybrid model are in its capacity to model the extreme tail (large fluctuations) behaviour using the EVT methods while also capturing conditional heteroscedasticity (changing variation) in the data through the GARCH framework. By implementing a dynamic method for forecasting a 1-day-ahead VaR, which combines the GARCH models and EVT to examine the extreme behaviour of major economic stock indices before and during the outbreak of the COVID-19 pandemic, Omar et al. (2020) and Paul and Sharma (2018) were able to accurately calculate VaR by using EVT methods. Explicitly, these authors adopted the GPD via the Peaks-overthreshold (POT) method. Comprehensive in-sample volatility modelling was implemented with skewed student distribution assumptions, and the information criterions were used to establish their goodness of fit. Furthermore, the VaR quantiles were estimated by using the conditional EVT (C-EVT) framework to obtain out-of-sample VaR forecasting results. The combined GARCH and EVT model performed relatively well in estimating the risk for all stock indices. The backtesting results demonstrate that the exponentialGARCH skewed-student’s-t and C-EVT models are the most appropriate techniques for better measuring and forecasting VaR in comparison with the conventional methods. Other studies like those of Echaust and Just (2020) examined the ability of value-atrisk estimates when each estimate is made with an optimal choice of the tails of the distribution by the combination of GARCH-EVT. Here, 5 methods were applied to describe the tail, namely the distance-metric method with the mean absolute penalty function, the minimisation of the asymptotic mean squared error (AMSE) estimate, the path-stability algorithm, the fixed-quantile procedure, and the automated eyeball 4K. MAKATJANE AND K. MMELESI method. The GARCH-EVT approach in combination with a novel algorithm to automatically determine the optimal threshold to model the tail distribution was proposed by Bruhn and Ernst (2022). Furthermore, individual market risks were aggregated with a t-student Copula to investigate possible diversification effects on a portfolio level (Hoffmann & Börner, 2020). The empirical analysis indicates that all examined cryptocurrencies show high volatility in their price movements, whereby Bitcoin acts as the most stable cryptocurrency. All returns distributions are heavy-tailed and subject to extreme tail risks. To effectively capture clustering volatility in the presence of structural breaks (structural changes) and tail behavior, Makatjane et al. (2021) also developed a two-stage analysis. The first stage was to estimate a Markov-switching exponential GARCH model to obtain regime switching residuals that are i.i.d. In the second stage, these authors applied EVT to the upper regime residuals and estimated both the GPD and the GEV dsitribution. This was to complete the study by McNeil et al. (2015), Sahamkhadam et al. (2018) and Echaust and Just (2020) among others. To manage the risks of a portfolio made up of commodities, currency indices, and equity securities, Koliai (2016) presented the GARCH-EVT with an R-vine copula model. The effectiveness of the generalised Lambda distribution (GLD), the generalised Pareto distribution, and the generalised extreme value distribution were advocated by Huang et al. (2017) and these authors simulated daily VaR and ES for log-returns of platinum, gold, and silver prices; giving GPD and GLD better performance over GEV distribution. The main approach to forecasting is the combination of several models (Hajirahimi & Khashei, 2019), which performs better than using just one model. Comparing the hybrid method to the single model, accurate performance can be produced (Büyükşahin & Ertekin, 2019). However, some studies like of Ibn Musah et al. (2018), for instance, aim to investigate the risks connected with the main Ghanaian stock exchange while combining EVT with artificial neural networks (ANNs). Recent research conducted by Ilyas et al. (2022) has proposed a new hybrid method, consisting of a fully modified Hodrick Prescott filter (FMHP) to improve prediction accuracy. This method consists of three main components: machine-learning-based prediction, novel features, and a noisefiltering technique. Moreover, the combination of generative adversarial networks (GANs) with extreme values proved to be more effective in modelling extreme values (Boulaguiem et al., 2022). The use of an ensemble learning-based rolling window approach to investigate the advantage of combining multiple GARCH-type models with LSTM (herein referenced Long-short Term-memory) to forecast value at risk is advocated by Kakade et al. (2022). These authors use coverage tests loss functions for the financial crisis of 2008–2009 and the COVID-19 recession of 2020–21 to assess the model’s performance on crude oil returns. When comparing the VaR forecasts of the hybrid models to the conventional and GARCH techniques, a notable improvement in quality and accuracy is noted. It is discovered that the most effective estimator of Value-at-Risk is the Filtered Historical Simulation technique. Moreover, Barrera et al. (2022) proposed a non-asymptotic convergence analysis of a two-step approach to learn value-at-risk and expected shortfall in a nonparametric setting using Rademacher and Vapnik-Chervonenkis bounds. The approach of these authors for VaR is extended to the problem of learning at once multiple VaRs JOURNAL OF APPLIED ECONOMICS 5 corresponding to different quantile levels. This led to the development of effective learning systems based on least-squares regressions and neural network quantiles. Without access to the latter, a posteriori Monte Carlo (non-nested) method was employed to estimate distances to the ground-truth VaR and ES. Numerical experiments in a Gaussian toy model and a financial case study, where the goal is to determine a dynamic initial margin, were used to demonstrate this. These numerical tests indicate that, despite initially seeming counter-intuitive, learning several quantiles (multi-α (I), multi-α (II), or multi-α (III)) can aid in effectively targeting extreme quantiles than a typical single quantile learning strategy. Recent research has established a flexible likelihood-based framework for the joint modelling of VaR and ES, based on the relationship between the quantile score function and the asymmetric Laplace density. Capturing the underlying combined dynamics of these two quantities is of great relevance in financial applications. To tackle this issue, Li et al. (2020) created a hybrid model that effectively captures the underlying dynamics of VaR and ES. The model is based on the asymmetric Laplace quasi-likelihood and uses the Long Short-Term Memory time series modelling technique from machine learning. This model is known as LSTM-AL. In the LSTM-AL model, these authors use the adaptive Markov chain Monte Carlo (MCMC) approach for Bayesian inference. Their empirical results show that the proposed LSTM-AL model has improved the VaR and ES forecasting accuracy over a range of well-established competing models. An early warning system for extreme daily losses for financial markets is of crucial modelling system. A three-stage methodology was established by Makatjane and Moroke (2021) where in the first stage a seasonal Autoregressive integrated moving average (SARIMA) was estimated to filter the series to obtain i.i.d residuals. By using the Markovchain Monte-Carlo approach, the Markov-switching exponential GARCH model coupled with generalised extreme value distribution was fitted to these i.i.d residuals and finally, the logistic model tree was time-honored to establish early warning signs. Alternatively, Makatjane and Tsoku (2022) overcome the dimensionality problem in forecasting VaR and ES uncertainty intervals in financial time series data by bootstrapping and backtesting density forecasts using Bayesian methods that are based on a weighted threshold and quantile of a continuously ranked probability score. These authors found that extension of this non-stationary distribution in literature is quite complicated since it requires specifications not only on how the usual Bayesian parameters change over time but also on those with bulk distribution components. This implies that the combination of a stochastic econometric model with extreme value theory procedures provides a robust basis necessary for the statistical backtesting and bootstrapping density predictions for VaR and ES. 2. Methodology A five-day financial time series exchange Johannesburg stock exchange/All-share index (FTSE/JSE-ALSI) for the period of 4 January 2010 to 5 April 2024 is used in this study. This consists of 3646 observations. The use of a five-day (s) frequency is based on the fact that, on weekends and holidays, stock markets are closed. Therefore, the South African Stock Exchange is not an exception. No trading is happening on these days, hence the recorded market prices are from Monday(s) – Friday(s); except where the holiday arrises 6K. MAKATJANE AND K. MMELESI during the week. In this case, only four data points are available instead of five. To avoid any exchange rate fluctuations, the index is kept in its original currency; i.e., the FTSE/ JSE-ALSI used in this study is kept in its ZAR; the South African currency. Let Xt be a stock price index on the day t and Xt1be a stock price index on the day t1. Letting rt to be stock returns at time t; Algieri and Leccadito (2020) showed that rt can be modeled by rt¼μtþεt(1) where μt is a time-varying mean of a time series and εt is the error term that should be modeled by εt¼ηtσt:(2) In model (2), σt is a time-varying dynamics of a time series; while ηt is an i.i.d process. 2.1. Proposed TBATS—generalised extreme value distribution The capacity of conventional seasonal and exponentially smooth models to handle dualcalendar, multi-seasonal, and noninteger seasonal time series is restricted (Sorokina et al., 2023). Several scholars have examined the challenge of handling intricate seasonal time series in this context, and to address this issue, the exponentially smooth model was a modified problem (Zhao & Zhang, 2022); hence the BATS model was introduced to address this complex time series pattern. The basic model form is expressed as BATS p;q;m1;���;mT ð Þ where B is the Box-Cox transformation for addressing heterogeneity, A is the ARMA error for addressing short-term dynamics, damping (if any) trends, and seasonal components, T and S are the trend and seasonal components of the time series respectively. In addition, p and q of the BATS model are the Autoregressive p and moving average q parameters respectively. Furthermore, m1;���;mT are the seasonal periods of the ARIMA model. The mathematical formula for the BATS model according to Munim (2022), is presented as follows rωð Þ t¼ rω t1 ω;ω�0 rt;ω¼0 �:(3) The major objective of the time series prediction technique known as the TBATS is to forecast complex seasonal trends by using exponential smoothing. Trigonometric seasonal functions were used in place of the seasonal components to develop the TBATS model, which is a modified time series method based on the BATS model (Thayyib et al., 2023). Thus, the structure of this model is the initial T that signifies the “trigonometric” function. This is expressed as TBATS ω;p;q;φ;m1 f g:���;mT;kT f gð Þ. Hence, Talkhi et al. (2024) showed that the mathematical formula for the TBATS model is as follows 2.2. Seasonal periods rωð Þ t¼lt1þϕbt1þXM i¼1Sið Þ tmiþdt(4) JOURNAL OF APPLIED ECONOMICS 7 their capital requirements and this is done by backtesting to cover market risk due to their trading activities. The capital requirement size according to is calculated as Cap ¼fact �max VaRt0:01ð Þ;1 60 X59 i¼1VaRt10:01ð Þ � � (23) where, Fact is the multiplication factor reported in Table 1. In other words, the required capital is equal to the multiplication factor that is multiplied by the highest value between today’s 99 percent VaR and the mean of the last 60 days 99 percent VaR. The risk manager should automatically upgrade the multiplication factor from 3.0 to 4.0. 3. Results and Discussion In this section of the study, we present the analysis and discussion of the results. These results are presented in tables and figures. Following Mushori and Chikobvu (2024), Chikobvu and Ndlovu (2023a), and Chinhamu et al. (2015), we analyse the gains and the losses separately because we want to model different risk profiles which leads us to understand the significance of a downside risk that is indicated by the negative returns (losses) and also understanding the tail behaviour of these losses which is crucial for risk management, particularly for setting capital reserves, insurance, and regulatory compliance. Moreover, positive returns (gains) highlight opportunities for substantial profits. Therefore, understanding the tail behaviour of gains can further help in devising investment strategies and optimisation of portfolios. It is worth noting that financial returns are asymmetric hence their distributions are often asymmetric, meaning that the tails on the left (losses) and the right (gains) can have different shapes and characteristics. Modelling them separately allows for a more accurate representation of their distinct behaviours. In addition, this approach helps to identify better and understand specific factors that drive positive and negative returns; where, negative returns might be driven by factors like market crashes, economic downturns, or unexpected negative news. However, positive returns might be driven by market booms, extraordinary company performance, or positive economic indicators. Finally, managing the risk of extreme losses is crucial for financial institutions, especially those involved in risk-sensitive activities like banking, insurance, and investment management. While on the other side, we want to identify and leverage the opportunities for extreme gains which are beneficial for investment strategies, hedge funds, and performance optimisation. Figure 1 depicts a plot of returns for FTSE/JSE-ALSI. Observed in this figure are the relative dynamics of the FTSE/JSE-ALSI index for the specified period. It is observed that Table 1. The basel II zones and the exceptions based on a 250 intraday trading sample. Confidence level Zone 90% 95% 99% Multiplication factor Green 0–32 0–17 0–4 3 Yellow 33–43 18–26 5–9 3.40, 3.50, 3.65, 3.75 or 3.85 Red �44 �27 �10 4 14 K. MAKATJANE AND K. MMELESI the mean of returns is not constant while the variance oscillates around high volatility (instability) and low volatility (tranquillity) with more extremes towards the end of the sample period. This implies that FTSE/JSE-ALSI have volatility clustering. Regarding the marginal distribution, the quantile-quantile (Q-Q) and normal histogram plot in Figure 2 reveal a strong departure from linearity in the lower and upper tails of the distribution. This evidence is also seen in Table 2 where the reported kurtosis is greater than three and the skewness is less than zero indicating that returns on FTSE/JSE-ALSI are asymmetrical with negatively skewed innovations. Figure 2. Normal histogram and normal quantile-quantile plots. Table 2. Descriptive statistics for returns series. Mean std deviation Skewness Kurtosis J-B test S-W test K-S test FTSE/JSE-Returns 0.000311 0.0096 −1.39970428 17.06483815 32932.373 (0.001) 0.898 (0.001) 0.484 (0.001) Note: NB: values in () are probability values of the S-W and A-D respectively. Figure 1. Five-day returns of FTSE/JSE-ALSI plot. JOURNAL OF APPLIED ECONOMICS 15 The normal histogram on the left panel of Figure 2 further confirms this negative skewness which also possesses high kurtosis that is above three. Therefore, a conclusion is that FTSE/JSE-ALSI returns have fat tail and leptokurtic behaviour. Khan et al. (2021) in their study of extreme value theory and COVID-19 have found fat tail behaviour in NIFTY-50 they have used. As evidenced in Table 2, the deduction here is that the overall returns on FTSE/JSEALSI are increasing during the period in question. The mean of the average returns is very small compared to the standard deviation. Mokoena (2016) and Beytell (2016) in their studies reported the same results. All in all, one can infer that the mean of returns is somehow smaller than the standard deviation of the returns. The FTSE/JSE-ALSI returns considered revealed kurtosis that is above 3, indicating that the returns series are all leptokurtic. Another feature of the returns series is the presence of skewness; they are found to be negatively skewed, implying that the distribution of FTSE/JSE-ALSI returns is significantly fatter than the Gaussian distribution. This negative skewness indicates that the lower tail of the distribution is thicker than the upper tail and declines in returns are more common than their increases. The Jarque-Bera (J-B), Shapiro Wilk (S-W), and Kolmogorov – Smirnov (K-S) tests confirmed non-normality at a 5% level of significance. The three test statistics led to the rejection of the null hypothesis of normality, confirming the results reported in Figure 2. According to Vee and Gonpot (2014) and Korkpoe and Junior (2018), stock markets are described by boom-bust cycles and such cycles feed into the volatility of the markets. 3.1. Trend analysis results Trends are evident in financial time series data during the week, month, quarter, etc. On weekends and holidays, all exchanges are closed; therefore, no trading takes place. The degree of activity on weekdays is significantly high. Across FTSE/JSE-ALSI, the day-ofthe-week impact is noticed in Figure 3. Generally, market activity is lowest on Monday and highest on the last two working days of the week. To be precise, trading activity starts growing progressively from Monday to Friday. On weekends and holidays, there are no Figure 3. Day of the Week pattern. 16 K. MAKATJANE AND K. MMELESI trading activities, leading to low opening stock prices on Monday. The weekend effect (also called the Monday effect, the day-of-the-week impact, or the Monday seasonal) describes how stocks tend to perform better on Fridays than on Mondays (Strohsal et al., 2019). In general, over all the years, there has been an increasing trend in FTSE/JSE-ALSI prices and this observation is seen in Figure A1 in Appendix 1 for the whole sample period. The Mann–Kendall test statistic and Sen’s slope estimator are used to analyse the longterm trends of the FTSE/JSE-ALSI. The outcome of the Mann–Kendall test results as reported in Table 3 and revealed that FTSE/JSE-ALSI has a significant monotonic increasing long-term trend because the value of τ>0. Sen’s slope value also shows significantly increasing magnitudes of trends, which correspond with the Mann– Kendall test results. The cause of this increase in the average returns for the other four days of the week is positive, while the average returns for Monday are noticeably negative, therefore, a significant positive trend is depicted in Table 3. According to Killian (2023) and Atsin and Ocran (2015), this increase is influenced by the demand that outstrips the supply. That is, more people want to buy the share price instead of selling it; and, consequently, the price rises. Otherwise, the price falls because supply is greater than demand. If more buyers move into the market, the demand grows, and share prices go up especially if there is limited supply. If supply and demand are just about equal, the share price is likely to move around in a narrow range for a while, until one of the factors outweighs the other (Khumalo, 2013). Other factors include macroeconomic factors such as interest rate changes, financial outlook, and inflation fluctuations. If the interest rate and inflation rate go down, and the economic outlook is in a good state, demand usually increases, and the share price is likely to rise (Killian, 2023). 3.2. Tbats-stationary generalised extreme value distribution To begin the main analysis, a TBATS is trained with a ratio of 75% training and 25% validation sets. To account for stationarity in our returns series, we employ the augmented Dickey-Fuller (ADF) and Philips Peron (PP). The same procedure is done by Chinhamu et al. (2015). The PP test is carried out using the Bartlett Kernel spectral estimating method, while the ADF test is set to lag 0 using the Schwartz Information Criterion (SIC). Table 4 presents the results of the ADF and PP tests, which show that the unit root null hypothesis is not supported for any test. As a result, it is possible to regard the return series on FTSE/ JSE-ALSI as stationary. Table 4. Results for ADF and PP unit root tests for FTSE/JSE-ALSI return series. Unit Root test Test statistic p-value ADF Test −13.696 0.001 Philips-Perron test −45.459 0.001 Table 3. Mann-Kendall test statistic and Sen’s slope estimator. M-K Test Statistic Kendall’s Tau p-value Sen’s Slope FTSE/JSE-ALSI 58.055 0.753 0.001 12.635 JOURNAL OF APPLIED ECONOMICS 17 Using the returns on FTSE/JSE-ALSI which are computed using model (1), a TBATS model is fitted to these returns. Assortment of the best TBATS model automatically is accomplished by using the TBATS function in Python for these returns. Employing the maximum likelihood estimation, the following parameter estimates are obtained: ^ λ¼0:337515;^ α¼0:021854 ^ β¼0:960208 the damping parameter estimate is ^ ϕ¼  0:91980288. 3.3. Selection of block minima/maxima After fitting our TBATS model, we extract the residuals which are i.i.d. As a first step, 25% of the residual series is reserved for future testing. The time interval of the data set to be separated is from 21 August 2020 to 5 April 2024. In the second step, the data set consisting of 3646 observations is divided into blocks of different sizes. Unlike Chikobvu and Ndlovu (2023a) who extracted monthly period minima/maxima from the daily returns of the BTC/USD and ZAR/USD returns data, we determine the minimum block size as 52 (i.e., weekly) and the maximum block size as 59 because this is the tradeoff problem between the bias and variance, and Figure 4 presents these results. Table 5 shows the first two blocks resulting from partitioning the residuals into 52 blocks by date to provide a detailed overview of the block partitioning. After receiving all the blocks, the maximum and minimum values of the blocks are calculated as shown in Table 5, and different data sets of maximum and minimum values are obtained. Figure 4 further shows a graph of all minimum and maximum values obtained from 52 blocks. As shown in Figure 4, the two variables derived from the minimum and maximum waves behave in a similar structure for both positive returns and negative returns. When a block is established, two random variables are calculated from the minimum and maximum values obtained from that block, and the characteristics of their distributions are recorded by determining how these variables are dispersed or best distributed. The block length ranges from 20 to 59 for negative returns and 18 to 59 for positive returns (owing to data sparsity), as determined by which block size is better explained or projected. The blocks allotted for the Figure 4. Block maxima (left panel) and block minima (right panel). 18 K. MAKATJANE AND K. MMELESI residuals from the TBATS are used to construct the data blocks with the numbers 20, 21, and 22 for negative returns and 18,19 and 20 for positive returns. Aside from the maximum and maximum values of the generated block sizes, the column totals for the block sizes of each of the 212 negative returns data and the 210 positive returns data are calculated starting from 20 to 59 and 18 to 59, respectively. As a result, it is possible to match the best and worst outcomes among the chosen blocks. But otherwise, the outcomes show which blocks in this applied analysis are the worst and the best. We therefore found that the block size achieving the best prediction for the estimation method obtained from the maximum (minimum) values that is, positive returns is 15 with rank 3 and for the negative returns is 11 with rank 4. These results are reported in Table 6. Since the best block sizes are obtained for both negative and positive returns, we now fit separately these gains (maxima) and losses (minima) to the GEV distribution and Table 7 shows the parameter estimates together with their corresponding standard errors (SE) for both gains and losses. It can be seen in Table 7 that for the negative returns (i.e., losses), the positive shape parameter is established, and the negative shape parameter is further established for positive returns (i.e., gains). This implies that negative returns or losses for the FTSE/JSE-ALSI are best modelled by a type II FréchetGEV distribution, while the gains are being modelled by a type III Weibull-GEV distribution. In a study comparing the riskiness of BitCoin/US Dollar and South African rand/US Dollar returns, Chikobvu and Ndlovu (2023a) fitted the GVE distribution to both the losses (negative returns) and gains (positive returns). The results of these authors portrayed a positive shape parameter for both returns, indicating a type II Fréchet-GEV which is a contrast to this study which found a mixture of family distributions for both gains and losses of FTSE/JSE-ALSI. Table 5. FTSE/JSE-ALSI sample application. Minima Maxima Size Date Block Size Date Block 1 08/02/2010 2246 1 04/03/2010 2546 2 08/62/2010 2200 2 01/02/2011 2835 3 08/08/2011 2481 3 16/05/2013 4967 4 24/06/2023 4133 4 22/10/2013 5500 5 30/01/2014 4666 5 10/04/2015 8101 6 16/01/2015 6343 6 15/08/2016 6979 7 11/12/2015 4703 7 20/03/2017 7242 8 06/07/2016 5674 8 20/03/2018 9620 9 06/12/2016 5944 9 28/08/2018 8119 10 27/06/2018 6717 10 15/01/2019 8469 11 26/10/2018 5952 11 20/06/2019 8109 12 14/08/2019 6886 12 20/11/2019 8408 13 23/03/2020 4544 13 29/07/2020 6505 14 30/10/2020 4727 14 12/03/2021 6405 15 23/04/2021 5496 15 15/06/2021 6500 16 26/11/2021 5270 16 30/02/2022 7232 17 06/07/2022 5086 17 01/03/2023 5989 18 15/12/2022 4740 18 01/03/2024 7433 19 11/05/2023 5141 20 03/042024 6439 Minimum 2200 Minimum 2546 Maximum 6886 Maximum 9620 JOURNAL OF APPLIED ECONOMICS 19 3.3.1. Goodness of fit test After model estimation, the goodness of fit (GoF) test is assessed. The Anderson-Darling and Shapiro–Wilk test for GoF tests are used and the results are presented in Table 8. Nonetheless, Stephens (1977) recommended these tests for the GoF for extreme value distributions, hence their use in this study. Chikobvu and Chifurira (2015) and Maposa et al. (2016) also used these tests for testing GoF for the GEV and GPD they estimated in their studies. The null hypothesis is that the returns following a GEV distribution are not rejected (p-values are not significant). Hence, the conclusion is that the FTSE/JSE-ALSI returns are modelled very well with the specified distribution. The estimated asymptotic laws can replace the distribution of extremes with the empirical distributions of the tails by these results. In extreme market conditions, it is thus possible to estimate the potential losses for stock and commodity indexes. Table 6. Best and worst block size ranking maximum FTSE/JSE-ALSI. Minimas Maximas Ranking Differnce Block size Ranking Differnce Block size 1 922 18 1 920 18 2 873 20 2 873 20 3 988 11** 3 977 22 4 889 15 4 889 15** 5 900 25 5 910 25 6 915 19 6 915 19 7 880 26 7 889 26 8 899 17 8 899 17 9 950 9 9 942 9 10 935 16 10 935 16 11 942 18 11 922 18 12 879 30 12 809 30 13 895 22 13 805 30 14 925 10 14 905 10 15 905 8 15 955 8 16 932 5 16 932 5 17 911 9 17 931 9 18 919 13 18 929 13 19 882 14 20 879 12 Table 7. Maximum likelihood estimates for the GEV distribution. Negative Returns (losses) Positive Returns (Gains) Extremes 11 15 ^ �1.156 −0.266 se ^ � �� 0.231 0.049 ^ σ−0.033 0.024 se ^ σð Þ 0.028 0.084 ^ μ0.0208 0.0084 se ^ μð Þ 0.031 0.015 Table 8. Goodness of fit test for GEV distribution. Test Positive Returns Negative Returns Anderson Darling Statistic 0.415 0.551 P-Value 0.335 0.156 Shapiro Wilk Statistic 0.062 0.074 P-Value 0.362 0.247 20 K. MAKATJANE AND K. MMELESI 3.3.2. Comparative analysis and combined forecast model using ensemble methods The purpose of this section is to determine the model that best mimics the data and also produces fewer forecasts. This will help in assisting the maximum dispatching of the South African stock market. The three error metrics namely mean absolute error (MAE), mean square error (MSE), and mean forecast error (MFE) are used to measure the performance of each model and the results are summarised in Table 9. Some tentative conclusions are drawn from this table, which indicates that no model is superior to the other. For positive returns, MSE selects the best as the TBATS while for negative the selected model is GEV using the same MSE metric. However, MAE selects GEV as the best model for positive returns while MFE selects both models. Because we found that no model is superior to the other, we therefore combine the two models and assess the performance of the combination. We determine the ideal weight to aggregate base learner predictions in such a way that the resulting ensemble minimises the total expected prediction error; hence, the aim is to find the most effective way to combine predictions. We, therefore, use the generalised ensemble method (GEM), stacking ensemble, blending, bagging, and boosting. These ensemble methods are chosen because they offer various advantages such as improved performance, reduced overfitting, model robustness, and flexibility in model selection and combination, making them popular choices in machine learning for achieving better results. Each ensemble method has its strengths and it is chosen based on the specific characteristics of the problem and the data at hand. Table 10 shows the average results of TBATS-GEV based on the MLE search method along with the mean squared error, mean absolute error, and mean forecast error made by each ensemble method. The superiority of the designed ensemble technique is seen by Table 10. Average results and created ensembles on negative and positive returns. Ensemble Method Positive Returns Negative-Returns TBATS-GEV MSE GEM 6.01 3.52 MSE Stacking 4.01 3.27 MSE Blending 5.31 3.45 MSE Bagging 6.15 4.87 MSE Boosting 7.28 6.88 TBATS-GEV MAE GEM 5.87 3.88 MAE Stacking 3.01 3.1 MAE Blending 7.31 5.45 MAE Bagging 8.15 7.87 MAE Boosting 7.13 6.19 TBATS-GEV MFE GEM 7.89 4.82 MFE Stacking 4.00 2.22 MFE Blending 5.31 5.01 MFE Bagging 7.25 4.87 MFE Boosting 7.28 6.88 Table 9. Performance Model selection criteria. MSE MAE MFE Positive Returns TBATS 8.779 7.238 6.328 GEV 8.953 6.998 6.328 Negative Returns TBATS 8.879 7.028 6.879 GEV 8.053 7.028 6.128 JOURNAL OF APPLIED ECONOMICS 21 comparing their forecasting errors. This demonstrates improvements in the stacking ensemble algorithm as compared to other ensemble algorithms. It is evident from Table 10 that the stacking ensemble outperformed other ensemble methods, by producing fewer forecasting errors; because, MSE, MAE, and MFE all advocate for the TBATS-GEV model that is combined using the stacking ensemble algorithm. We, therefore, proceed to estimate the market risk using TBATS-GEV combined with the stacking ensemble algorithm. 3.4. Risk estimates Since the stacking ensemble algorithm has outperformed other ensemble algorithms, the four risk measures discussed in section 2 are computed using the TBAT-GEV distribution to evaluate the risk of losses and gains in the financial sector in South Africa for returns on five day FTSE/JSE-ALSI and the results are presented in Table 11. The computed values suggest that the losses (i.e., negative returns) are riskier than the gains (negative returns) since they have higher values of risk measures. For the gains, at 99%, the VaR value is 0.000850. This implies that there is a lower likelihood of significant losses; indicating that the risk of large negative returns is relatively low with a value of 0.85% for VaR at a 99% level of significance. The 99% level of significance suggests that lower risk is realised for the four estimated risk measures than at 90% and 95%. This is because, at a higher level of significance, less room for error is tolerated. These results are in contradiction of the one found by Chikobvu and Ndlovu (2023b); because the study by these authors had found the lower VaR estimate at 95% level. It is worth noting that at 99% level, Glue-VaR is negative giving the value of − 0.000012235. This suggests that if one is focusing on the lower tail of the distribution of FTSE/JSE-ALSI returns, there is a high likelihood of investors incurring losses at this tail of the distribution. This negative value reflects a relatively high level of risk. It indicates that the investment has the potential for significant losses, especially in extreme market conditions or during events that negatively impact the investment. Yu et al. (2019) also revealed that a one-day change in the financial market’s value would not decrease by more than 0.00123%. To take into account market liquidity constraints and Basel regulations, 5-day risk horizons in addition to the more typical 1-day horizon are considered. Comparing losses to gains, the results indicate that the prospects of potential extreme losses are greater than the prospects of potential extreme gains. 3.4.1. Evaluating the accuracy of risk measures A backtesting procedure is used to evaluate the risk measures’ accuracy, and the estimates are shown in Tables 12 and 13. While backtesting the estimated risk measures, a predicted loss is compared to the actual loss, observed the following day. An exceedance Table 11. Computation risk measures. Positive Returns Negative Returns P VaR ES CTE Glue-VaR VaR ES CTE Glue-VaR 0.9 0.001544 0.023454 0.023854 0.000630 0.001839 0.027366 0.027366 0.001110169 0.95 0.001230 0.023924 0.023924 0.000363 0.001471 0.027331 0.027331 0.000432336 0.99 0.000850 0.023967 0.023967 0.000134 0.000734 0.027366 0.027366 −0.000012350 22 K. MAKATJANE AND K. MMELESI occurs when the actual loss exceeds the calculated risk measure. An expected number of exceedances over the VaR curve is predicted statistically for VaR. VaR0:99, Rydell (2013) stressed that the loss exceeds that value 99 times out of 100, and one exceedance for every 100 observations is statistically expected. These exceedances are used as the reference. For this study, the combined TBATS-GEV is considered reliable if the number of exceedances when backtesting the model is within a 10% confidence interval from the statistically expected losses. The null hypothesis is that the risk model is correctly specified and accurately predicts financial risk for FTSE/JSE-ALSI. Nevertheless, both the Kupiec and the Christoffersen tests suggest that the null hypothesis is not rejected and the conclusion is that, the four risk measures produced reliable, efficient, and unbiased risk estimates for both 95% and 99% confidence levels. For long-time periods such as more than ten years as is the case in this study, the combined model produces suitable risk estimates. According to Bee and Trapin (2018), these long periods are indicated by high probability values for losses and gains at all selected confidence levels of all the tests used. The results of the regulatory backtest on the risk measures are presented in Table 13. The numbers in the table display how many times each risk measure in each zone has been backtested over the course of a year (250 trading days). The test is ranked as follows: (1) the green zone if there are 0 to 4 exceptions reported within a 250-trade day window; (2) the yellow zone if there are 5 to 9 exceptions; and (3) the red zone if there are more than 9 exceptions. All the risk measures fall into the yellow zone, which imposes a fine ranging from 0 to 85 times the risk measure to calculate the market risk charge. Interestingly, the hybrids with 250 days of data performed uniformly throughout all the risk measures. This gives an annual average penalty that ranges from 20% to 59%. Generally, all the risk measures showed that the combined TBATS-GEV is a good Table 13. Basel III zones backtest results. Zone 250 Days VaR 250 Days ES 250 Days CTE 250 Days Glue VaR Gains Green Zone 3 3 5 6 Yellow Zone 2 2 1 3 Red Zone 2 1 0 1 average annual penalty 0.487 0.543 0.655 0.654 Losses Green Zone 5 3 4 4 Yellow Zone 3 1 1 3 Red Zone 1 1 1 0 average annual penalty 0.472 0.376 0.55 0.35 Table 12. Backtesting risk measures. p-values for Kupiec test p-values for Christoffersen test Risk Measure Level 0.95 0.99 Risk Measure Level 0.95 0.99 GEV VaR Gains 0.517 0.378 VaR Gains 0.207 0.885 GEV ES Gains 0.919 0.877 ES Gains 0.865 0.987 GEV CTE Gains 0.517 0.522 CTE Gains 0.476 0.939 GEV Glue-VaR Gains 0.796 0.95 Glue-VaR Gains 0.871 0.994 GEV VaR Losses 0.529 0.902 VaR Losses 0.801 0.739 GEV ES Losses 0.668 0.872 ES Losses 0.701 0.885 GEV CTE Losses 0.65 0.872 CTE Losses 0.698 0.839 GEV Glue-VaR Losses 0.694 0.682 Glue-VaR Losses 0.98 0.794 JOURNAL OF APPLIED ECONOMICS 23 Appendix Figure A1. Annual trend for each year. 30 K. MAKATJANE AND K. MMELESI