Forecasting Korean Stock Price Index (KOSPI) using back propagation neural network model, Bayesian Chiao's model, and Sarima model.

Lee, Kyung Joo ; Chi, Albert Y. ; Yoo, Sehwan 等

In this study, we forecast Korean Stock Price Index (KOSPI) using historical weekly KOSPI data and three forecasting models such as back-propagation neural network model (BPNN), a Bayesian Chiao's model (BC), and a seasonal autoregressive integrated moving average model (SARIMA). KOSPI are forecasted over three different periods (i.e., short-term, mid-term, & long-term). The performance of the forecasting models is measured by the forecast accuracy metrics such as absolute forecasting errors and square forecasting errors of each model.

The findings are as follows: First, between BPNN and BC, BPNN performs better than BC for mid term and long term forecasting, while BC performs better than BPNN for the short term forecasting. Second, between BPNN and SARIMA, SARIMA performs better than BPNN for mid term and long term forecasting, while BPNN does better than SARIMA the short term forecasting. Between SARIMA and BC, SARIMA performs better than BC for mid term and long term forecasting, while the other way around is true for the short term forecasting.

In sum, the SARIMA performs best among the three models tested for mid term and long term forecasting, while BC performs best for the short term forecasting.

INTRODUCTION

The ability to forecast the capital market price index is critical to individual investors, institutional investors, and financial analysts. Among many forecasting models for stock prices and market price index, the seasonal autoregressive integrated moving average model (SARIMA) has been one of the most popular forecasting models in capital market studies.

Recently, the neural network model has been frequently used in many capital market studies (e.g., Ansari et. al. (1994), Hamid et. al. (2004), Huang et. al. (2005), Kumar et. al. (2006), Malik et. al. (2006), Stansell et. al. (2004), and Trinkle et. al. (2005)). Major reasons for the neural network model's popularity in capital market forecast are twofold. First, the neural network model is data driven method which learns from sample data and hence does not require any underlying assumptions about the data. Thus, the model is known as a universal functional approximate without severe model misspecification problems due to wrong assumptions (Hornik et. al. (1989)). The model is also outstanding in processing large amount of fuzzy, noisy, and unstructured data. For example, Hutchinson et. al. (1994) examine stock option price data and show that the neural network model is computationally less time consuming and more accurate non-parametric forecasting method, especially when the underlying asset pricing dynamics are unknown or when the pricing equation cannot be solved analytically. Second, stock price data are large, highly complex and hard to model because the pricing dynamics are unknown, which suits the neural network model.

The Bayesian Chiao's model (BC) may be another powerful and practical tool to forecast capital market data for the following reasons. First, the main thoughts of the BC is the dynamic way of combining the prior information (i.e. either from historical datasets or from previous subjective experience) with the current observations, during the process of posterior information. Second, most of the traditional statistics applications are based on the assumptions of independent, identical distributed (i.e. i.i.d.) normal random variables. However, the merit of the BC is to assume independence of variables only. No more identical distributions are needed. Third, the pros of BC can be the dynamic adaptive mechanism of integrating prior knowledge and the current information for accurately predicting the immediate future outcomes. However, the con of this model is its high dependence on the quality of the initial values of the estimates. Further, without constantly absorbing realistic datasets, the long term iterative predictions of this model perform poorly, if merely repeatedly applies the BC. Thus, in general, the BC is more effective in short-term forecasting than it is in long-term forecasting.

Korean stock market is considered more volatile than its US counterpart and hence has fuzzier and unstructured price data, which suits the Neural Network model and the BC well (for the short term forecasting, in particular). Thus, it is meaningful endeavor to examine how well the neural network model and the BC perform in forecasting more volatile Korean market data relative to a conventional SARIMA model which is one of the most popular forecasting models in capital market studies. The purpose of this study is to compare the ability of the neural network model, the BC, and SARIMA model in forecasting Korean Stock Price Index (KOSPI). Weekly data of KOSPI are analyzed in this study.

The remainder of this paper is organized as follows. Sample data and methodology are discussed in the next section, followed by discussions on empirical results in section three. The concluding remarks are presented in the final section.

DATA AND METHODOLOGY

Index Data

The data used in this study are KOSPI for closing prices from the Korean Stock Exchange (KSE) data base. The data series span from 4th January 1999 to 29th May 2006, totaling 390 weeks (89 months) of observations. Although KOSPI data is available since the opening of the Korean Options Exchange for stock price index in July 1997, we exclude two year's data (1997-1998) because the Korean stock market had suffered the severe financial crisis (IMF crisis) during this period.

The data are divided into two sub-periods, one for the estimation and the other for the forecasting. We use four different forecasting periods to examine the potential impact of forecasting horizons on the forecasting accuracy. Forecasting horizons used are 20% (long range), 13% and 8% (mid range), and 6% (short range) of the total number of observations. For the long range forecasting, the first 313 weekly data are used for model identification and estimation, while the remaining 77 weekly data (about 20% of 390 weeks) are reserved for evaluating the performance of SARIMA, BC, and the neural network model. Other forecasting periods were defined in the similar way, resulting in mid range (31-50 weeks ahead) and short range (23 weeks ahead) forecasting horizons.

Neural Network Forecasting

In this study, the back-propagation neural network model (BPNN) is used for time series forecasting. The main reasons for adopting BPNN are twofold. First, BPNN is one of the most popular neural network models in forecasting. Second, BPNN is an efficient way to calculate the partial derivatives of the networks error function with respect to the weights and hence to develop a network model that minimizes the discrepancy between real data and the output of the network model.

BPNN can be trained using the historical data of a time series in order to capture the non-linear characteristics of the specific time series. The model parameters will be adjusted iteratively by a process of minimizing the forecasting errors. Fore time series forecasting, the relationship between output (yt) and the inputs (yt-1 yt-2, ... , yt-p) can be described by the following mathematical formulae.

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1)

where [a.sub.j] (j = 0,1,2, ... ,q) is a bias on the jth unit, and [w.sub.ij] (i = 0,1,2, ... ,p; j = 0,1,2.,,,q) is the connection weights between layers of the model, f(*) is the transfer function of the hidden layer, p is the number of input nodes and q is the number of hidden nodes. The BPNN performs a nonlinear functional mapping from the past observation ([y.sub.t-1] [y.sub.t-2], ... [y.sub.t-p]), to the future value ([y.sub.t]), i.e.,

[y.sub.t] = [phi]([y.sub.t-1] [y.sub.t-2], ... [y.sub.t-p]) + et (2)

where w is a vector of all parameter and o is a function determined by the network structure and connection weights.

The NeuroSolutions NBuilder toolbox is used to train data for model developments and test data for forecasting accuracy of the models developed. Here the stop criteria for the supervised training of the networks are specified as follows. The maximum epochs specify how many number of iterations (over the training set) will be done if no other criterion kicks in. The training terminates when one of the following four conditions is met (1) mean square error of the validation set begins to rise, indicating that over fitting might be happening.; (2) when the threshold is less then 0.01, i.e., we are on effectively flat ground; (3) when training time has reached 1000 epochs; (4) the goal (difference between output and target is less than 0.01) has been met. After the training was completed, its epochs and the simulation procedure completed successfully, which indicated the network was trained was predicting the output as desired.

According to the principle of Ockham's razor, the simplest networks topology yielding satisfactory results is used. The networks are created as '2-1-1': that is, two input layers, one hidden layer, and one output layer. The network is also trained using various other topologies such as 2-X-1, while X = 2, 3, 4, and 5. However, the best results are obtained when there is one hidden layer (i.e., X=1).

Bayesian Chiao Forecasting

With respect to the sequence of the logistic data having two possible results (i.e. pass, which means "[X.sub.m-1] < [X.sub.m]"; or failure, which means "[X.sub.m-1] < [X.sub.m]), the trends of the central tendency and deviation can be sequentially adjusted. From BC, the posterior distribution [X.sub.m+1] can be calculated as the following:

"Pass Case": If the predicate value of [X.sub.m] is "pass", then

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (3a) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (3b)

"Failure Case": If the predicate value of [X.sub.m] is "failure", then

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (4a) [2.sub.m+1] = [2.sub.m](1 -[(1 + [a.sup.-2.sub.g][m.sup.-2]).sup.-1] f(D)[(f(D)/F(D))+D]/F(D)) (4b)

Where f(D) = the value of probability density function of [X.sub.m] = D; F(D) = the value of cumulative density function of [X.sub.m] = D; [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] the square root of ([[sigma].sup.2.sub.m] + [a.sup.-2.sub.g] A = [c.sub.g] + (1 - [c.sub.g]) F (-D); [a.sub.g] = the parameter for the degree of similarity on the attribute values; [b.sub.g] = the parameter for the degree of proximity; [c.sub.g] = the parameter for the degree of uncertainty in the estimation; [[micro].sub.m] = the prior central tendency of the observed values of [X.sub.m]; m = the prior variance of the observed values of [X.sub.m]; [[micro].sub.m+1] = the posterior central tendency of the observed values of [X.sub.m+1]; m+1 = the posterior variance of the observed values of [X.sub.m+1]; [X.sub.m] = the prior random variable; [X.sub.m+1] = the posterior random variable;

Time-series Forecasting

To obtain the KOSPI forecasts from the SARIMA, we adopted the Box and Jenkins' method. Basically, Box and Jenkins' method uses the following three-stage approach to select an appropriate model for the purpose of estimating and forecasting a time-series data.

Identification: we used the SARIMA procedure in SAS statistical software to determine plausible models. The SARIMA procedure uses standard diagnostics such as plots of the series, autocorrelation function (ACF), inverse autocorrelation function, and partial autocorrelation function (PACF).

Estimation: Each of the tentative models is fit and the various coefficient estimates are examined. The estimated models are compared using standard criteria such as Akaike Information Criteria and the significance level of coefficients.

Diagnostic checking: SARIMA procedure is used to check if the residuals from the different models are white noise. The procedure uses diagnostics tests such as ACF, PACF, and Ljung-Box Q-statistics for serial correlation.

Applying these steps, SARIMA (110)(12) was selected as the best-fitting forecasting model for weekly KOSPI data.

Measurement of Forecast Accuracy

Forecast error (FE) is determined by subtracting forecasted value from actual value of KOSPI, and then deflating the difference by the absolute value of actual data as follows:

FEt = (At - Ft)/|At| (5)

where At = actual value of KOSPI in period t. Ft = forecasted value of KOSPI in period t.

The reason for using the absolute value of actual KOSPI as deflator is to correct for negative values. Accuracy of a forecast model is measured by sign-neutral forecast error metrics such as absolute forecast errors (AFE) and squared forecast errors (SFE). Since the results are essentially the same, we report only those from using AFE.

RESULTS

Descriptive statistics of forecast accuracy (AFE) from BPNN, BC, and SARIMA using weekly KOSPI price data are presented in Table 1. Those statistics of AFE for four different forecasting horizons such as 77 weeks ahead (long), 50 weeks ahead (upper middle), 31 weeks ahead (lower middle), and 23 weeks ahead (short) are presented in Table 1. Mean, standard deviation, minimum, and maximum of AFE for four different forecasting horizons are also presented in Table 1. The SARIMA provides smallest mean AFE for forecasting horizons of 31 weeks ahead or longer, while BC provides the smallest mean AFE.

This may indicate that, among the three models, SARIMA is the most effective in mid-term and long-term forecasting while BC is the best in short-term forecasting, which is consistent with the findings of Wang et. al. and our prediction.

The mean AFE from the above-mentioned three forecasting models, Kruskal-Wallis [chi square] statistics, and the corresponding p-values are presented in Panel A of Table 2. Kruskal-Wallis [chi square] statistics of all four forecasting horizons are statistically significant at 0.001 significance level, indicating that, overall, forecasting errors from the three models are significantly different each other.

Results from pair wise comparisons between BPNN and BC, between BPNN and SARIMA, and between BC and SARIMA are presented in Panel B of Table 2.

Comparisons between BPNN and SARIMA show that SRIMA produce smaller forecasting errors than BPNN for mid-term and long-term forecasting horizons (i.e., 31 weeks, 50 weeks, & 77 weeks), while BPNN produce smaller forecasting errors than SARIMA for short-term forecasting horizon (i.e., 23 weeks). All differences in forecasting errors between BPNN and SARIMA are statistically significant at the significance level of 0.01. This indicates that SARIMA performs better than BPNN in mid-term and long-term forecasting while BPNN performs better than SARIMA in short-term forecasting.

Comparisons between BPNN and BC show that BPNN produce smaller forecasting errors than BC for 77 weeks and 50 weeks forecasting horizons. And the differences in forecasting errors between the two methods are statistically significant at the significance level of 0.01. This indicates that BPNN produce more accurate forecasts than BC for these relatively longer-term forecasting. On the other hand, BC produce smaller forecasting errors than BPNN for 31 weeks and 23 weeks forecasting horizons but the differences in forecasting errors between the two models are statistically significant only for 31 weeks forecasting horizon at the significance level of 0.05. This may indicate that BC performs better than BPNN in 31 weeks ahead forecasting but no meaningful conclusion can be drawn for the 23 weeks ahead forecasting.

Comparisons between BC and SARIMA show that SRIMA produce smaller forecasting errors than BC for mid-term and long-term forecasting horizons (i.e., 31 weeks, 50 weeks, & 77 weeks), while BC produce smaller forecasting errors than SARIMA for short-term forecasting horizon (i.e., 23 weeks). All differences in forecasting errors between BC and SARIMA are statistically significant at the significance level of 0.01. This indicates that SARIMA performs better than BC in mid-term and long-term forecasting while BC performs better than SARIMA in short-term forecasting.

In sum, SARIMA is the best forecasting model for mid-term and long-term forecasting, while BC is the best forecasting model for short-term forecasting.

CONCLUSIONS

The purpose of this study is to compare the forecasting performance of back-propagation neural network model (BPNN), a Bayesian Chiao's model (BC), and a seasonal autoregressive integrated moving average model (SARIMA) in forecasting weekly Korean Stock Price Index (KOSPI). Forecasting performance is measured by the forecast accuracy metrics such as absolute forecasting errors (AFE) and square forecasting errors (SFE) of each model.

KOSPI data over the 390 week period extending from January 1999 to May 2006 are analyzed. We find the followings: First, the SARIMA provides most accurate forecasts among the three models tested for mid term and long term forecasting, while BC provides the most accurate forecasts for the short term forecasting. Second, between BPNN and BC, BPNN provides more accurate forecasts than BC for mid term and long term forecasting, while insignificant difference in forecasting errors exists between the two models for the short term forecasting.

These results are robust across different measures of forecast accuracy. Since the accuracy of forecasting values is dependent on the developing process of forecasting models, the results of this study may also be sensitive to the developing process of the BPNN, BC, and SARIMA.

REFERENCES

Ansari, A. M. and Frank C. Lin (1994). Forecasting financial trends in stock markets with neural networks. Institute of Electrical and Electronics Engineers, pp188-196.

Chiao, T. (2006) Strategic Quality Competitiveness Management; International Journal of Information Technology and Management, Vol. 5, Nos. 2/3, 2006, pp. 197-211.

Hamid, S. and Z. Iqbal (2004). Using neural networks for forecasting volatility of S & P 500 Index future prices. Journal of Business Research, Vol. 57-10, pp 1116.

Hornik, K., M. Stinchcombe, & H. White (1989). Multi-layered feedforward networks are universal approximators. Neural Networks, Vol. 2, pp.359-366.

Huang, W., Y. Nakamori, and S. Wang (2005). Forecasting stock market movement direction with support vector machine. Computers & Operations Research Vol. 32-10, pp 2513.

Hutchinson, J., W. Andrew, & t. Poggio (1994).A nonparametric approach to pricing and hedging derivative securities via learning networks. Journal of Finance, Vol. 49. pp. 851-889.

Kumar, K. and S. Bhattacharya (2006). Artificial neural network vs. linear discriminant analysis in credit ratings forecast: a comparative study of prediction performances. Review of Accounting & Finance Vol. 5-3, pp 216.

Malik, F. and M. Nasereddin (2006). Forecasting output using oil prices: a cascaded artificial neural network approach. Journal of Economics and Business Vol. 58, pp 168-180.

Medeiros, M., T. Terasvirta, and G. Rech (2006). Building neural network models for time series: a statistical approach. Journal of Forecasting Vol. 25-1, pp49.

Stansell, S. and S. Eakins (2004). Forecasting the direction of change in sector stock indexes: An application of neural networks. Journal of Asset Management, Vol. 5-1, pp 37-48.

Tang, y., F. Xu, X. Wan, and Y. Zhang (Working paper). Web-based fuzzy networks for stock prediction. Georgia State University.

Trinkle, B. (2005). Forecasting annual excess stock returns via an adaptive network-based fuzzy inference system. Intelligent System in Accounting, Finance, and Management, Vol. 13-3, pp 165.

Wang, W., D. Okunbor, and F. Lin (Working paper). Future trend of the shanghai stockmarket. University of Maryland-Eastern Shore.

Kyung Joo Lee, University of Maryland Eastern Shore

Albert Y. Chi, University of Maryland Eastern Shore

Sehwan Yoo, University of Advancing Technology

John Jongdae Jin, University of Maryland-Eastern Shore Table 1: Descriptive Statistics for Forecast Accuracy Forecasting Model Mean Std Dev Min Horizon 77 BPNN 0.123 0.090 0.001 BC 0.224 0.123 0.003 SARIMA 0.099 0.063 0.002 50 BPNN 0.171 0.073 0.022 BC 0.202 0.086 0.006 SARIMA 0.108 0.052 0.006 31 BPNN 0.099 0.027 0.062 BC 0.093 0.038 0.002 SARIMA 0.055 0.026 0.004 23 BPNN 0.040 0.025 0.004 BC 0.031 0.016 0.001 SARIMA 0.095 0.066 0.010 Forecasting Quartiles Max Horizon 25% 50% 75% 77 0.039 0.109 0.207 0.274 0.110 0.234 0.341 0.398 0.047 0.102 0.151 0.214 50 0.113 0.201 0.227 0.273 0.135 0.241 0.267 0.313 0.062 0.123 0.145 0.216 31 0.078 0.089 0.127 0.148 0.072 0.083 0.127 0.156 0.034 0.056 0.074 0.102 23 0.014 0.049 0.065 0.077 0.019 0.037 0.045 0.059 0.035 0.084 0.125 0.252 1) Forecast errors are defined as: (At-Ft)/|At|, where At=actual value of KOSPI index, Ft=forecast value of KOSPI index. Forecast accuracy is defined as the mean absolute forecast error (AFE). 2) The number of weeks used to estimate forecasting models is different depending on forecasting horizons. For example, 313 weeks (367 weeks) of data were used to obtain forecasts for the 77 weeks (23 weeks) ahead. Each forecasting horizon represents roughly 20%, 13%, 8% and 6% of total number of observations (390 weeks). 3) BPNN= Back-propagation neural network model without hidden layer. BC = Bayesian Chiao's model. SARIMA= Best-fitted SARIMA model: (110)(12). Table 2: Comparison of Forecast Accuracies across Forecasting Models Panel A: Summary Statistics for Forecast Accuracy and Overall Comparisons Forecasting Models Forecasting Horizon BPNN BC (weeks) 77 0.123 0.224 50 0.171 0.202 31 0.099 0.093 23 0.040 0.031 Panel B: Multiple Pair wise Comparisons Forecasting Horizon (weeks) BPNN VS. SARIMA BPNN VS. BC 77 0.024 (5.44) *** -0.101 (21.90) *** 50 0.062 (11.70) *** -0.032 (14.26) *** 31 0.044 (6.69) *** 0.006 (2.20) ** 23 -0.055 (3.25) *** 0.009 (1.18) Forecasting Models Forecasting Kruskal-Wallis 2 Horizon SARIMA stat (p-value) (weeks) 77 0.099 43.698 (0.001) *** 50 0.108 33.902 (0.001) *** 31 0.055 28.500 (0.001) *** 23 0.095 17.701 (0.001) *** Panel B: Multiple Pair wise Comparisons Forecasting Horizon (weeks) BC VS. SARIMA 77 0.126 (15.65) *** 50 0.094 (13.63) *** 31 0.037 (4.59) *** 23 -0.064 (5.09) *** 1) Forecast errors are defined as: (At-Ft)/|At|, where At=actual value of KOSPI index, Ft=forecast value of KOSPI index. Forecast accuracy is defined as the mean absolute forecast error (AFE). 2) NN= Back-propagation neural network model without hidden layer. BC= Bayesian dynamic adaptive model. SARIMA= Best-fitted SARIMA model: (110)(12). 3) Matched-pair t-tests were used for multiple pair wise comparisons. ***: Significant at <0.01; **: significant at <0.05; *: significant at <0.10;