Forecasting volatility for financial assets has recently garnered significant interest among investors and professional analysts because of its importance in risk management, derivative pricing, and portfolio allocation. In recent years, availability of high frequency intra-day asset price data sets has enabled us to realize the hidden volatility via realized volatility (RV). Since volatility is related to future asset price uncertainty, forecast of RV is one of the main issues. Diverse methodologies for forecasting RV have been developed during the last decade in various academic fields and applied in real markets. Major results on RV forecasting were reviewed by McAleer and Medeiros (2008a), Andersen and Terasvirta (2009), and Shin (2018).

These days, taking well advantage of hidden nonlinearity in time series data, deep learning has been widely considered in regression and classification problems, and has proved to perform very well compared with other methods. Furthermore, since neural network methods do not require normality assumptions for population distribution, financial analysts, economists and statisticians are increasingly using these methods for data analysis (Mantri et al., 2010). With this trend, there have been many research works which compare methods using deep neural networks with others for time series data. For example, there were comparisons between the performances of artificial neural network (ANN) and autoregressive integrated moving average models for forecasting commodity prices (Kohzadi et al., 1996); ANNs and support vector machines were used to predict prices on the Istanbul stock exchange national 100 index (Kara et al., 2011); Mantri et al. (2010) calculated the volatilities of Indian stock markets using the artificial networks and the statistical time series methods of GARCH, EGARCH, GJR-GARCH, IGARCH models.

The main characteristic of time series data is serial dependence. In particular, financial volatility data sets such as RV data sets and implied volatility data sets show very strong persistent serial dependence, called long-memory, see Section 2 and for example Engle and Patton (2007), Andersen and Terasvirta (2009), and Bollerslev et al. (2016). The serial dependence of RV data sets leads us to adopt recurrent neural networks (RNNs), particularly long short-term memory (LSTM), a variant of RNN, which has recently emerged as an effective model for a variety of applications involving sequential data (Hochreiter and Schmidhuber, 1997; Karpathy et al., 2015). In addition, gated recurrent unit (GRU) is widely used for analyzing time series data including language, video, and others, also can outperform LSTM units in terms of convergence in CPU time, parameter updates, and generalization (Chung et al., 2014; Cho et al., 2014). There is also bidirectional RNN, an extended version of RNN, which can be trained without limitations using input information just up to a preset future frame and offers better performance than RNN. The hyperparameters of the recurrent network with LSTM units and GRU such as the input length for serial dependence consideration, the number of layers, and others are chosen by the nested cross validation (CV).

In the recent literature, RNN models have been sucessfully adopted for forecasting volatilities of financial time series. Kim and Won (2018) considered a combination of the deep learning LSTM for volatility forecast of the KOSPI (Korean stock price index) and compared the proposed model with other standard models. Liu (2019) compared volatility forecasts of the US S&P 500 index and AAPL (Nasdaq Apple stock price) by the SVM, the LSTM and the statistical GARCH model and identified the situations in which the LSTM has better performance. Bucci (2020) studied forecast of monthly realized volatility of the US S&P stock price index and compared some RNN models and some econometric models. Lei et al. (2021) considered the LSTM with text mining as a sentiment analysis to improve forecast of the realized volatility of Stock Rambler in China stock market. They used the comments of investors in the stock bar as the basis for the sentiment analysis. We see some RNN methods for forecasting volatilities of commodity prices: The CNN-LSTM by Vidal and Kristjanpoller (2020) for gold price and the LSTM+GARCH model by Hu et al. (2020) for copper price. In the above RNN methods, they do not address the dominant data feature of long-memory and asymmetry of financial volatility data sets. The RNN models would produce better volatility forecasts if the long-memory and asymmetry features of volatility are well addressed. These features will be addressed by ratio transformation and piecewise min-max normalization. This consideration will be shown to produce considerably better RV forecasts for 10 major world stock price indices than other methods which ignore the dominant data features.

We forecast daily RV of major stock price indices of the US and the EU. Recall that the deep learning forecasts usually show out-of-scale issue that forecasted values are limited within the range of training data, and the issue is generally resolved by considering ratios rather than data themselves (Fisher and Karauss, 2018; Liu and Mehta, 2019; Oztekin et al., 2016). Moreover, compared to original RV’s, the ratioed RV’s has substantially less persistent serial dependence which render the RNN forecast model to perform better. We therefore consider relative ratio of adjacent RVs instead of the RV itself. Learning and forecasting are first made for the RV ratios, and forecasts of RV values are next computed from those of RV ratio values. The forecasts based on ratioed RVs will be shown to be better than those based on RV themselves.

In addition to the long-memory feature of RV data sets, another important one is asymmetry as reviewed by Corsi (2009), McAleer and Medeiros (2008a), and many others. The asymmetric feature of RV data sets still remains in RV ratio data sets: Heavier right tails than left tails, as shown in Sections 2 and 5. We address the asymmetry issue by piecewise min-max (PM) normalization to scale the data. The PM normalization is shown to faithfully address the asymmetry in the RV ratio data resulting in better forecasts than the other usual normalizations: Min-max (MM) normalization, Gaussian mixture (GM) normalization.

Superiority of the proposed method is illustrated in an out-of-sample forecast comparison for RV’s of 10 major US and EU stock price indices: S&P 500, Russell 2000, DJIA, NASDAQ, FTSE 100, CAC 40, DAX, AEX, SSMI, IBEX 35. (i) The proposed RNN model with ratio transformation and PM normalization (RNN-R-PM) is the best among 6 RNN models (with or without ratio transformation and MM, GM, PM other normalization) and 4 benchmarking models of the AR, the SVM, the DNN, and the CNN. (ii) The RNN is shown to be better than the DNN and the CNN when applied to the original RV with the MM normalization as is the usual practice. (iii) Ratio transformation of RV gives the RNN better forecasts than original RV. (iv) PM normalization yields better forecast for the RNN than MM and GM normalizations.

The rest of the paper is organized as follows. Section 2 describes our research data. Section 3 presents three normalization methods, RNN-based methods, and the nested CV. Section 4 explains experimental design, evaluation measures, an implementation of the nested CV method. Section 5 provides an illustration of the modeling procedure, the CV results and forecast performance comparison results of the RNN-based methods and the benchmarking models. Finally, Section 6 concludes this work.

A realized variance is simply the sum of squares of intra-day log-returns for which the unit of time is usually one day and it is indexed by t = 1, 2, . . . . For a day t, if we let equally spaced intra-day log price observations be p_{t−1+γ j} for j = 0, 1, . . . , M, γ = 1/M, γ-spaced realized variance for day t is given as Σj=1M(rtj(γ))2 for t = 1, 2, . . . , where rtj(γ)=pt-1+γj-pt-1+γ(j-1) is the log-return for the γ-spaced intraday interval [t − 1 + γ( j − 1), t − 1 + γ j]. Then, RV for a day t is given by vt=Σj=1M(rtj(γ))2. We consider RV data sets of 10 major stock price indices, 4 in the US and 6 in the EU for the period from Jan. 5, 2004 to Nov. 30, 2017 which are obtained from the oxford-man institute’s realized library (https://realized.oxford-man.ox.ac.uk/). sampling interval for the RVs is γ = 5/(24 × 60) day, i.e., 5 minute. The total number of RV values for each asset is T = 3344.

Summary statistics of the RV data sets are shown in Table 1. The means of RV are all roughly 1% and skewness values are all much larger than 0 indicating very right-skewed data distributions, which can also be confirmed by much larger values of (max - median) than (median - min) values, for example 29.66 − 0.62 > 0.62 − 0.11 for S&P. The CoV denotes the coefficient of variation, SD/Mean. As the randomness limits the accuracy of forecasts in general, forecasting with less dispersed data is presumably advantageous, and the CoV can be utilized as a rough indicator for this. The CoV of RV has a value around 0.5 for every index.

From Table 1, it can be inferred that the RV data is somewhat irregular due to high skewness and CoV. Also, it is generally reported that the RNN-based forecasts usually suffer from the out-of-sacle issue that the forecast value tends to be limited within the range seen from the training data. These concerns can be relieved by using the relative ratio of RV instead of the original value to forecast (Liu and Mehta, 2019). The relative ratio of RV is denoted u_t = v_t/v_t−1, t = 1, . . . , T − 1. Since RV values are positive, the relative ratio of adjacent values of RV is defined well. When we compute a forecast v̂_t+1 of v_t+1 at time t, we first conducted forecast û_t+1 of u_t+1 using a trained model for u₁, . . . , u_t and next construct forecast v̂_t+1 = û_t+1v_t. Summary statistics of the ratio data are reported in Table 2. We can check from the relatively small values for the skewness and CoV than those of RV that the relative ratio of RV is less dispersed and skewed than RV. These advantageous stochastic features support taking relative ratios instead of RV values for forecasting.

In addition to avoiding the out of scale issue and having smaller skewness and CoV than the RV data, the ratioed RV data has another important advantage of resolving the long memory of the original RV data, which considerably improve deep learning forecasts. For this discussion, we choose the S&P 500. The conclusion of this paragraph holds good for all the other assets. Figure 1 shows time series plot of the daily RV series v_t of the S&P 500 and its autocorrelation function (ACF) ρ^kv=corr^(vt,vt-k), k = 1,. . .,200. The figure reveals that the RV series v_t has persistent strong autocorrelation even for large k, for example, ρ^10v=0.65,ρ^100v=0.29 which is called long memory. A long memory of v_t means that two observations (v_t, v_t−k) have non-negligible correlation even though they are far apart, i.e. corr(v_t, v_t−k) is not close to 0 even for large k. This means that v_t’s have high similarity for a wide range of t. The long-memory of v_t is a consequence of the repeated “∧”-like peaks as shown in Figure 1(a), which span for long periods of more than several weeks or several months. When a batch of consecutive v_t’s are input to an RNN, information in the batch is not well utilized by the RNN because of high similarity of neighboring observations in the batch, producing poor forecasts. The high similarity of neighboring values arising from long memory is considerably resolved in the ratio data u_t = v_t/v_t−1 as shown in its Figure 2(a) and autocorrelation function ρ^ku=Corr^(ut,ut-k) in Figure 2(b). In Figure 2(b), ρ^ku, k = 2, . . . , 200, all lie within or slightly out of the dotted ±0.035 band, which is the ±2 × standard error (ρ^ku)=±2/T=±0.035. Note however ρ^1u=-0.34 is significantly non-zero, telling that u_t has a short-memory. Now, a batch of consecutive u_t in the ratioed RV consists of less autocorrelated values than v_t of the original RV and would be more well utilized in RNN forecast model to produce better forecasts as will be revealed in Section 5.3.

The remaining non-negligible autocorrelation for one lag k = 1 (significantly nonzero ρ^1u) as well as significant asymmetry (skewness > 0) as shown in Table 2 lead us to consider forecasting by the RNN, which is more suitable for serially dependent data than other deep learning models such as the DNN and the CNN. In our RNN methods, we will address the asymmetry by PMM normalization and the serial correlation by the LSTM and the GRU.

3.1. Normalization

The values passed between layers in the RNN models need to be scaled to [0, 1] or [−1, 1]. Normalization techniques can be used to scale the data and can speed up training time by starting the training process with the scaled data (Liu and Mehta, 2019). Since we focus on forecasting the relative ratio of RV, normalization is adopted to it. The parameters required for the normalization process are usually obtained from a set of values, which is mostly a train set. Here, let be the set on which normalization is based. In this work, three kinds of normalization techniques are employed as described in the following.

The min-max normalization (MMN) is the simplest method to adjust range of data to lie in [0, 1] (Liu and Mehta, 2019). For a value u, the MMN is applied as follows

It preserves the stochastic properties of the data except that only the range is scaled to [0, 1]. Also, the MMN has the advantage of low complexity of denormalization.

The Gaussian mixture normalization (GMN) exploits a density estimation method using linear superposition of Gaussian distributions. The distribution of values in can be approximated through Gaussian mixture model. We use a well-known fact that F_X(X) follows a uniform distribution on (0, 1) for any random variable X whose distribution function F_X is continuous. The GMN transforms a value u to be in (0, 1) as

where Φ(u∣μm,σm2)’s are the distribution functions of the Gaussian distributions N(μm,σm2). The parameters π_m, μ_m and σ_m are usually estimated using the expectation maximization algorithm (Bishop, 2006). This technique enables learning independent of stochastic characteristics such as the skewness of data. However, the GMN has the disadvantage of complexity of denormalization because the inverse of Gaussian mixture requires numerical computations. In this work, M = 3 is used.

The Piecewise min-max normalization (PMN) is a simple combination of the two techniques mentioned above in that the distributional feature of data is addressed by different min-max scaling to the left and right half of data. For a value u, the PMN is applied as follows

where I{·} is an indicator function. The asymmetric features of an RV data set are more faithfully addressed by the PMN than the MMN and the GMN because, in the PMN, different normalizations are applied to the left half and the right half of the RV data set. This normalization transforms the median of data to 1/2 and we can retrieve the median of the data from the forecast value of 1/2 through denormalization. As in the case of the MMN, it has also the advantage of low complexity of denormalization.

3.2. RNN methods

We use RNNs which are widely used for sequential data (Karpathy et al., 2015). Computations in most RNN’s are composed of a sequence of 3 blocks having parameters to be updated and associated transformations: Block 1, from the input to the hidden state; block 2, from the previous hidden state to the next hidden state; and block 3, from the last hidden state to the output. To construct a recurrent network of this structure, hyperparameters should be determined: The input length, the number of time steps of an input sequence; the hidden size; the number of units of a hidden layer; the number of layers, and others. In practical applications, gated RNNs are used as highly effective sequence models, which handle the issue of vanishing or exploding derivatives by creating paths through time. For a cell of a gated RNN, we consider two structures LSTM and GRU in this work. In addition, a bidirectional structure that depends on the whole input sequence by using not only the forward causal structure but also the backward direction can also be considered (Schuster and Paliwal, 1997). As the structure of the network varies according to the cell structure and the RNN direction, these are also considered as hyperparameters (Goodfellow et al., 2016). A method for hyperparameter determination is discussed in Section 3.3

Here we briefly explain the RNN model we use. Let the input length be given as q. For an input sequence (x₁, . . . , x_q) of length q, which are normalized ratios through one of the MMN, the GMN, or the PMN mentioned in Section 3.1. The x_t at t is also represented by ht0 for later use. Although a boldface is used here, it is actually a scalar in this work. We denote the hidden state vectors as htl∈ℝn, where t is a time step, l = 1, . . . , L is the index of layer, and n is the hidden size. The model consists of a total of L +1 layers, with L hidden layers and 1 output layer. Each element of the time series data is injected into the network one at a time sequentially. Although the detailed structure for calculating the hidden states htl of the hidden layers is different for each cell structure, it is common in that ht-1l, from the previous time of the same layer, and htl-1, from the previous layer at present time, are used as

where W^l is the parameter matrix of the l^th hidden layer. A detailed description of the transformation g(·) in each cell structure and bidirectional recurrent network are given in Appendix A. For the last hidden state hqL, we apply an additional layer to this n-dimensional vector to get an estimate x̂_q+1 of the normalized ratio of RV at time q + 1. The operation of the output layer can be expressed as x^q+1=sigm(WOhqL), where W_O ∈ ℝ^1×n is the parameter matrix. Sigmoid activation ‘sigm’ guarantees that the estimated value is in between 0 and 1. Then, the parameter matrices are updated in the way of minimizing the mean squared error (MSE) loss between x̂_q+1 and x_q+1 over all the train data.

3.3. Nested CV for hyperparameter determination

The RNN-based methods discussed in the previous subsection have various hyperparameters, such as the RNN direction, the input length, the number of layers, and others that need to be specified. It is important to choose good hyperparameter combination for training an RNN-based model for the good forecast. In order to compare the forecast accuracies of two or more methods with different hyperparameters, researchers tend to use CV (Oztekin et al., 2016). Typically, the data for training an RNN-based model and evaluating forecast performance consist of a train set and a test set. A train set is further divided into a train subset and a validation subset. Since there are temporal dependencies in the data, in order to accurately simulate the real world forecast environment in which we stand in the present and forecast the future (Tashman, 2000), we need to set the validation set comes after the train subset, and the test set after the train set. Also, if we ignore the time sequence and choose a test set arbitrarily, then our test set error is a poor estimate of error on an independent test set. For these reasons, traditional k-fold CV is inappropriate when dealing with time series data. Instead, we use the ‘Nested CV’ procedure which provides an almost unbiased estimate of the true error (Varma and Simon, 2006): We create multiple train/test splits and average the errors over all the splits to produce a better estimate of model forecast error. Day forward-chaining technique which is based on method called forward-chaining or rolling-origin evaluation (Tashman, 2000) is used in this study. A detailed implementation of the CV is described in Section 4.4.

We elaborate how the methods described in Section 3 are applied to our data. For the purpose of forecasting future RV values, we perform CV in exactly the same way it would be done in the actual forecast process. That is, the models are trained on past values and their performances are tested on future values. This type of CV can be used not only to determine hyperparameters but also to examine the effects of normalization techniques through the stochastic characteristics inherent in the data.

Let denote the dataset that will be used for tuning hyperparameters. First, the dataset () is divided into subsets consisting of d elements. If we assume that J subsets denoted by D₁, D₂, . . . , D_J are used, the time index of the data is adjusted so that the elements of D_j become {v_(j−1)d+1, . . . , v_jd}. The RV values before the first element v₁ of D₁ are naturally indexed with values below zero. We assume that one subset serves as a test set in the CV process. When we want to evaluate forecast performance with a subset as the test set, we configure a train subset and a validation subset preceding the test set so that the sizes of train subset and validation subset have a ratio of α_train : α_valid where α_train and α_valid are positive integers. The validation subset is used for adopting early stopping technique (Goodfellow et al., 2016). In the following we discuss in more detail how the RNN model training is performed and the CV process is applied. The methods of this section as well as those in Section 3 are illustrated in Section 5.1 in a forecasting setup for RV data.

4.1. Data for training

We describe how to construct input-output pairs for training our RNN-based models discussed in Section 3.2. Let a subset D_k for a test set be given, where k is larger than α_train + α_valid. Then, α_train +α_valid consecutive subsets preceding D_k are utilized for training. When the length q of the input data sequence for an RNN-based model is given, we use a set of q + 1 + (α_train + α_valid)d consecutive RV values prior to v_(k−1)d+1, which is the first value of D_k, as

to construct a set of q + (α_train + α_valid)d consecutive RV ratio values as

Now, the normalization is based on . We apply one of the normalization techniques given in Section 3.1 to the values in , and make a set of normalized ratio values as

Finally, we construct input-output pairs for RNN learning by sliding the window of length q + 1 by a time unit over the sequence of normalized relative ratios. Then, we get (α_train + α_valid)d input-output pairs as

Among them, the first α_traind input-output pairs are used as a train subset for actual update of the parameter matrices of RNN model and the last α_validd pairs are used as a validation subset for adopting early stopping technique.

4.2. Forecasting process

We describe how forecasts are made using a trained RNN-based model. Let the subset D_k to be forecasted and the length q of the input are given. After the RNN-based model learning of Section 3.2 with the input-output pairs constructed as described in the previous subsection 4.1, for each element v_t+1 in D_k, we compute a forecast v̂_t+1 by using the trained model. We begin with the previous q + 1 RV values (v_t−q, . . . , v_t). Then, the sequence (x_t−q+1, . . . , x_t) of the normalized ratios is generated by applying the same normalization technique as in the training process to the sequence of relative ratios. Specifically, for t − q + 1 ≤ s ≤ t where is the set of RV ratios used for training defined in the previous subsection 4.1. If an input (x_t−q+1, . . . , x_t) is applied to the trained model, an output x̂_t+1 is generated. To make a forecast for v_t+1, we first denormalize this value by u^t+1=f.-1(x^t+1;Uk,q). Then, our forecast value for v_t+1 is given by v̂_t+1 = v_tû_t+1.

4.3. Evaluation measures

We have 1-step forecasts {v̂_(k−1)d+1, . . . , v̂_kd} for {v_(k−1)d+1, . . . , v_kd} in D_k computed by the method of subsections 4.1 and 4.2. Then, forecast accuracies are usually compared by computing the measures given in the following:

4.4. Application of nested CV

We elaborate the CV process adopted in this work. To evaluate the forecast performance of the RNN-based method for a combination of hyperparameters, the forecast accuracy is measured on the last K subsets D_J−K+1, . . . , D_J . In other words, we put each of the last K subsets as a test set and evaluate the forecast performance through K different learning and forecast processes using the methods mentioned in subsections 4.1 and 4.2. This process is applied to actual forecasts in the same way, so that the CV simulates exactly what we want to do. The CV estimate of the overall performance criteria is calculated by simply the average of the forecast performances on the K consecutive subsets {D_J−K+1, . . . , D_J } as Ce=(1/K)Σj=J-K+1JFAj, where FA_j is a forecast accuracy such as MSE or those in (4.3)–(4.5) and others. The hyperparameters are determined to minimize C_e. This procedure is illustrated in Figure 3.

For each of possible hyperparameter combinations, we perform the above-mentioned process to get the CV estimate. Then, based on the results from this exhaustive CV process, the combination of hyperparameters to be used for the actual forecast is determined.

Recall that the total number of RV values for each asset is T = 3344. Hyperparameter tuning and forecasting are made blockwise with block length d = 150. Let an asset be given. Let D₀ be the block of the first 344 observations. For notational convenience, we use the time index so that v₋₃₄₃ be the first observation and D₀ = {v₋₃₄₃, . . . , v₀}. Let D₁, . . . , D₂₀ be the blocks of the size d of the remaining 3000 observations so that D₁ = {v₁, . . . , v_d}, . . . , D₂₀ = {v_19d+1, . . . , v_20d}. We wish to compare 1-step out-of-sample forecasts of the latest 450 RV values in D₁₈, D₁₉, D₂₀ by the proposed RNN methods and those by the benchmarking models of the classical AR model and the SVR (support vector regression), the DNN, and the CNN. We use J = 17 blocks D₁, . . . , D₁₇ for hyperparameter tuning. Since the forecast target has 450 elements, the test set consists of three latest blocks D₁₈, D₁₉, and D₂₀. To determine the hyperparameters for forecasting by the nested CV method in Section 4.4, we take K = 5 and utilize subsets D₁₃, D₁₄, . . . , D₁₇ as test data sets for the nested-CV. For convenience, we denote ∪j=1317Dj as D_CV. The sizes of train and validation subsets are set to α_traind = 10d and α_validd = 2d, respectively. The CV process gives three hyperparamter combinations for each normalization technique with which the RNN-model is developed for forecast. For each hyperparameter combination and a normalization technique, the weight W is first estimated by the deep learning models of LSTM or GRU described in Appendix A using data in D₁, . . . , D₁₇. This weight is used for computing 1-step out-of-sample forecast of elements in D₁₈. For forecasts of elements in D₁₉ and D₂₀, the weight W is newly estimated blockwise: For j = 1, 2, for forecasting elements in D_18+j, we use the weights newly estimated using data D_6+j, . . . , D_17+j. We now have all forecast values of the last 450 RV values in blocks D₁₈, D₁₉, and D₂₀, which are compared with the corresponding observed values by the evaluation measures RMSE, MAE, MAPE in Section 4.3. More detailed modeling procedure is illustrated in Section 5.1. The CV results and forecasted results are given in Section 5.2 and in Section 5.3, respectively.

5.1. An illustration of the modeling procedure

This subsection illustrates the methods in Sections 3, 4 in the setup of the first paragraph of Section 5. Recall that data lengths are all T = 3344. We first split the original RV data {v₋₃₄₃, . . . , v₃₀₀₀} into a sequence D₀, D₁, D₂, . . . , D₂₀ such that D₀ = {v₋₃₄₃, . . . , v₀}, D₁ = {v₁, . . . , v_d}, . . . , D₂₀ = {v_19d+1, . . . , v_20d}. Our aim is constructing 1-step forecasts of elements in D₁₈, D₁₉, D₂₀.

5.2. CV results

5.3. Forecast results

We perform forecasts for data subsets D₁₈, D₁₉, and D₂₀. The forecast results by the RNN based models and other benchmarking models are compared through various performance measures discussed in Section 4.3. As a benchmarking classical model, we consider the AR model of order p, AR(p), which is a statistical counterpart of the current RNN-based model:

where ϕ = (ϕ₀, . . . , ϕ_p)′ are the parameters of the model, and {ɛ_t}_t≥0 is a sequence of regression error. The autoregression order p is determined by the minimum Bayesian information criterion (BIC). Other important models of the support vector machine (SVM), the deep neural network (DNN), and the convolutional neural network (CNN) are also considered as benchmarking models. While the CNN was originally applied to image analysis, in many recent studies, the CNN has been applied to forecasting of time series such as stock price, volatility, and many others, see Vidal and Kristjanpoller (2020), Elalem et al. (2023), Liang et al. (2022), Choi and Shin (2022). This is because CNN captures local features of a time series and has a lower computational cost than the RNN. The DNN and the CNN are applied to the original RV data with the min-max normalization as is the common practice. The AR and the SVM are applied to the original RV as is the usual practice. For the SVM, we consider the radial basis function (RBF) as the kernel function. We also consider the kernel parameters C and γ for the SVM which are optimized over {0.01, 0.1, 10, 100} and {0.25, 1, 4}, respectively. The hyperparameters (q, L, n) for the DNN are chosen by the same method for RNN as in Section 5.2.2. For the CNN, the 1D-CNN with a size 3 kernel is considered, the number of filter is optimized over {8, 16, 32, 64}, and the number of layer is optimized over {2, 3}. The RNN models with original RV are also compared. The AR model and the SVM update the coefficients every time when an additional observation is available, while the RNN, the DNN, and the CNN update weight matrix W every 150 times when all the additional observations in the block of length 150 are available.

For the RNN methods with ratio transformation, forecasts are made for RV elements in D₁₈, D₁₉, and D₂₀. For each asset, all of the best 3 hyperparameter combinations given in Tables B.1–B.3 are used for forecast. For each hyperparameter combination, we perform forecast five times. Then, as an ensemble method, the final forecast results are made by averaging the total of 15 forecast results, five forecasts from each of the three hyperparameter combinations. Forecasts by other models are constructed similarly. Tables 4–6 show forecast results for the forecasting performance criteria of MAPE, MAE, and RMSE. In the tables and discussions below, for simplicity, we write “O” and “R” for orignal RV and ratioed RV, respectively; “MM”, “GM”, “PM” for min-max, Gausssian mixture, and piecewise min-max, respectively, normalization. Therefore, for example, RNN-R-PM denote the RNN for ratioed RV with PM. Note that RNN-R-PM adopts our main strategy of resolving long-memory of and addressing asymmetry of RV data sets by R and PM, respectively. We discuss general findings from the results of Tables 4–6.

• RNN-R-PM produces generally the best results. The superiority of the RNN-R-PM model to the other models holds strongly in the MAPE performance being best in all the 10 assets (Table 4) and somewhat strongly in the MAE performance being best in 6 assets out of 10 assets (Table 5). In the RMSE performance, even though RNN-R-MM tends to produce the best results, RNN-R-PM has also good RMSE performance not much worse than that of the best model as shown in Table 6. For all the 10 assets, the RNN-R-PM is considerably better than the benchmarking the AR, the SVM, the DNN, and the CNN models in all of MAPE, MAE, and RMSE with an exception of the AR for asset SSMI in RMSE.

• RNN-O produces much better forecasts than the DNN and somewhat better than the CNN except for 1 asset of the DAX out of 10 assets. Recall that RNN-O, the DNN, and the CNN all apply to the original RV data with the same MM normalization. Recall also that original RV data sets have long memory of persistent strong autocorrelation as discussed in Section 2. The RNN-O takes well advantage of the long memory of the RV data sets and produce better forecasts than the DNN and the CNN which do not explicitly address the serial dependence.

• Ratio transformation generally yields forecast improvement. In forecast performances of RNN-O and RNN-R, R is generally better than the corresponding O except for RMSE of RNN-R-PM model. For example, RNN-R-MM is better than RNN-O-MM. The better forecast performance of R than O is a consequence of resolved long memory in ratioed RV as discussed in Section 2.

• PM is better than MM and GM for the RNN. Superiority of PM over MM and GM holds both in RNN-O and in RNN-R in all of MAPE, MAE and RMSE with an excepton that, in RMSE performance for RNN-R, MM tends to be best but PM is not far away from the best. This better performance of PM than MM and GM is a result of better exploiting of the asymmetry in the RV or in the ratioed RV as in (2).

We considered recurrent neural network (RNN) forecast of realized volatility (RV) of 10 stock price indices, four from the US, and six from the Europe. We took three different normalization methods into the relative ratio of RV to adjust the range of data to fit the RNN based model. The piecewise min-max (PM) normalization is proven to faithfully address the asymmetric data distribution feature resulting in better forecasts than min-max or Gaussian mixture normalizations. Also two RNN-variant models LSTM, GRU and bidirectional of them are utilized. We tested the combinations to obtain the optimal models by applying the nested cross-validation (CV). The RNN-based model with PM normalization and ratio transformation (RNN-R-PM) outperformed other RNN models and the bench marking models of AR (autoregressive regression), the SVM (support vector machine), the DNN (deep neural network), and the CNN (convolutional neural network) model in most of the data sets. In particular, the RNN-R-PM performed substantially better than other models in all 10 the data sets in terms of MAPE. Forecasts via ratioed RV data were shown to be better than those based on RV themselves.

Here, we briefly introduce the RNN structures based on the framework provided in Karpathy et al. (2015). We denote the hidden state vectors as h t l ∈ ℝ n, where t = 1, . . . , q is regarded as time, l = 1, . . . , L is the depth and n is the hidden size. The model consists of L hidden layers and 1 output layer with a total of L + 1 layers.

Cell structures for gated RNNs

Although detailed structure for calculating the hidden states h t l s of the hidden layers is different for each model, it is common in that h t - 1 l, which is from the previous time of the same layer, and h t l - 1, which is from the previous layer at present, are used. In the following, parameter matrices are denoted by W with the corresponding subscript and superscript.

LSTM

LSTM, one of variants of RNN, is designed to prevent gradient vanishing and exploding problems in RNN. This model has cell states and uses input, forget, and output gates for each time step to control what information to throw away from and store in the cell state and update the old cell state. The detailed expression of the steps in LSTM is as follows.

( i t l f t l o t l g t l )   = ( sigm sigm sigm tanh )   W l   ( h t l - 1 h t - 1 l ) , c t l   = f t l ⊙ c t - 1 l + i t l ⊙ g t l , h t l   = o t l ⊙ tanh ( c t l ) ,

where i t l , f t l , o t l , g t l ∈ ℝ n, W^l ∈ ℝ^4n×2n, and ⊙ means element-wise product. In this work, since h t 0 is a scalar, W¹ ∈ ℝ^4n×(n+1). Also, the activation functions, sigmoid function ‘sigm’ and ‘tanh’, are applied element-wise. Note that ‘sigm’ and ‘tanh’ have ranges of [0, 1] and [−1, 1], respectively. The three vectors i t l , f t l and o t l act as input, forget, and output gates which are used to control the cell state. Sigmoid activation derives values from 0 to 1 to keep the model differentiable. The g t l vector through tanh acts as a new candidate, with a value between −1 and 1, and reflected to cell state as vector i t l adjusts. The structure of the LSTM solves the long-term dependency problem in that the gradient of the cell state uninterrupted during backpropagation even if the length of the input sequence is long.

GRU

GRU introduced by Cho et al. (2014) is a simpler model than the standard LSTM model. It combines the forget and input gates into a single ‘update’ gate, also it merges the cell state and hidden state. The detailed expression of the steps in GRU is as follows.

( r t l z t l ) = ( sigm sigm )   W r l   ( h t l - 1 h t - 1 l ) , h ˜ t l = tanh ( W x l h t l - 1 + W g l ( r t l ⊙ h t - 1 l ) ) , h t l = ( 1 - z t l ) ⊙ h t - 1 l + z t l ⊙ h ˜ t l ,

where r t l , z t l ∈ ℝ n ,   W r l ∈ ℝ 2 n × 2 n ,   W g l , W x l ∈ ℝ n × n. For the same reason as in the case of LSTM, W r 1 ∈ ℝ 2 n × ( n + 1 ) and W x 1 ∈ ℝ n × 1.

Figure A.1

Considered structures for RNN model.

Bidirectional RNN

A bidirectional RNN connects two hidden layers of opposite directions to the same output. With this structure, it is possible to get information from backward and forward states which do not have any interactions each other simultaneously (Schuster and Paliwal, 1997). The structure of a bidirectional model is illustrated in Figure A.1(b). We call a typical RNN model described above a unidirectional model, which is illustrated in Figure A.1(a), to distinguish it from a bidirectional model. Specifically, in the bidirectional model, bidirectional structure is applied to the bottom L − 1 layers except the last layer.

According to the CV errors, we take the best three hyperprameter combinations for RNN-R, which are applied to ratioed RV, and these are given in Tables B.1–B.3.

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2023-00239009, 2022R1F1A1068578, RS-2023-00242528).