Forecasting and Empirical Methods in Finance and Macroeconomics.

Diebold, Francis X.

Francis X. Diebold [*]

All economic agents forecast all the time, and forecasting figures especially prominently in financial and macroeconomic contexts. Central to finance, for example, is the idea of expected present value of earnings flows, and central to macroeconomics is the idea of expectations and their effects on investment and consumption decisions. Moreover predictive ideas in finance and macroeconomics are very much intertwined. For example, modern asset pricing models attribute excess returns and return predictability in part to macroeconomic factors such as recession risk.

In finance recently, there has been extensive inquiry into issues such as long-horizon mean reversion in asset returns, persistence in mutual fund performance, volatility and correlation forecasting with applications to financial risk management, and selection biases attributable to survival or data snooping. [1] In macroeconomics, we have seen the development and application of new coincident and leading indicators and tracking portfolios, diffusion indexes, regime-switching models (with potentially time-varying transition probabilities), and new breeds of macroeconomic models that demand new tools for estimation and forecasting.

The development and assessment of econometric methods for use in empirical finance and macroeconomics, with special emphasis on problems of prediction, is very important. That is the subject of my own research program, as well as of an NBER working group that Kenneth D. West and I lead. [2] Here I describe some aspects of that research, ranging from general issues of forecast construction and evaluation to specific topics such as financial asset return volatility and business cycles.

Forecast Construction and Evaluation in Finance and Macroeconomics

Motivated by advances in finance and macroeconomics, recent research has produced new forecasting methods and refined existing ones. [3] For example, prediction problems involving asymmetric loss functions arise routinely in many fields, including finance, as when nonlinear tax schedules have different effects on speculative profits and losses. [4] In recent work, I have developed methods for optimal prediction under general loss structures, characterized the optimal predictor, provided workable methods for computing it, and established tight links to new work on volatility forecastability, which I discuss later. [5]

In related work motivated by financial considerations, such as "convergence trades," and macroeconomic considerations, such as long-run stability of the "great ratios," Peter F. Christoffersen and I have considered the forecasting of co-integrated variables. We show that at long horizons nothing is lost by ignoring co-integration when forecasts are evaluated using standard multivariate forecast accuracy measures. [6] Ultimately, our results suggest not that co-integration is unimportant but that standard forecast accuracy measures are deficient because they fail to value the maintenance of co-integrating relationships among variables. We suggest alternative measures that explicitly do this.

Forecast accuracy is obviously important because forecasts are used to guide decisions. Accuracy is also important to those who produce forecasts, because reputations and fortunes rise and fall with their accuracy. Comparisons of forecast accuracy are also important more generally to economists, as they must discriminate among competing economic hypotheses. Predictive performance and model adequacy are inextricably linked: predictive failure implies model inadequacy.

The evaluation of forecast accuracy is particularly common in finance and macroeconomics. In finance, one often needs to assess the validity of claims that a certain model can predict returns relative to a benchmark, such as a martingale. This is a question of point forecasting, and much has been written about the evaluation and combination of point forecasts. [7] In particular, Roberto S. Mariano and I have developed formal methods for testing the null hypothesis: that there is no difference in the accuracy of two competing forecasts. [8] A wide variety of accuracy measures can be used (in particular, the loss function need not be quadratic, nor even symmetric), and forecast errors can be non-Gaussian, non-zero mean, serially correlated, and contemporaneously correlated. Subsequent research has extended our approach to account for parameter estimation uncertainty [9] and data snooping bias. [10]

Recent developments in finance and financial risk management encourage the use of density forecasts: forecasts stated as complete densities rather than as point forecasts or confidence intervals. However, appraisal of density forecasts has been hampered by lack of effective tools. In recent work with Todd A. Gunther and Anthony S. Tay, I have developed a framework for rigorously assessing the adequacy of density forecasts under minimal assumptions. I have used the new tools to evaluate a variety of density forecasts involving both simulated and actual equity and exchange rate returns. [11]

Most recently, Jinyong Hahn, Tay, and I have extended the density forecast evaluation methods to the multivariate case. [12] Among other things, the multivariate framework lets us evaluate the adequacy of density forecasts in capturing cross-variable interactions, such as time-varying conditional correlations. We also provide conditions under which a technique of density forecast "calibration" can be used to improve density forecasts that are deficient. We show how the calibration method can be used to generate good density forecasts from econometric models, even when the conditional density is unknown.

Density forecast evaluation methods are also valuable in macroeconomic contexts, as my recent work with Tay and Kenneth F. Wallis demonstrates. [13] Since 1968, the Survey of Professional Forecasters has asked respondents to provide a complete probability distribution of expected U.S. inflation. Evaluation of the adequacy of those density forecasts reveals several deficiencies. The probability of a large negative inflation shock is generally overestimated. And, in more recent years, the probability of a large shock of either sign is overestimated.

Modeling and Forecasting Financial Asset Return Volatility

Volatility and correlation are central to finance. Recent work has clarified the comparative desirability of alternative estimators of volatility and correlation and has noted the attractive properties of the so-called realized volatility estimator, used prominently in the classic work of Robert Merton, Kenneth French, and others. Realized volatility is trivial to compute. Further, we now know that under standard diffusion assumptions, and when using the high-frequency underlying returns now becoming widely available, realized volatility is effectively an error-free measure. Hence, for many practical purposes, we can treat volatilities and correlations as observed rather than latent.

Observable volatility creates entirely new opportunities: we can analyze it, optimize it, use it, and forecast it with much simpler techniques than the complex econometric models required when volatility is latent. My recent work with Torben Andersen, Tim Bollerslev, and Paul Labys exploits this insight intensively, in understanding both the unconditional and conditional distributions of realized asset return volatility, in developing tools for optimizing the construction of realized volatility measures, in using realized volatility to make sharp inferences about the conditional distributions of asset returns, and in explicit modeling and forecasting of realized volatility. [14]

Noteworthy products of the research include a simple normality-inducing volatility transformation, high contemporaneous correlation across volatilities, high correlation between correlation and volatilities, pronounced and highly persistent temporal variation in both volatilities and correlation, evidence of long-memory dynamics in both volatilities and correlation, and precise scaling laws under temporal aggregation. [15] The results should be useful in producing improved strategies for asset pricing, asset allocation, and risk management, which explicitly account for time-varying volatility and correlation.

Any such strategies exploiting time-varying volatility or correlation, however, require taking a stand on the horizon at which returns are measured. Different horizons are relevant for different applications (for example, managing a trading desk versus managing a university's endowment). Hence, related work involving volatility estimation and forecasting in financial risk management has focused on the return horizon. In a study with Andrew Hickman, Atsushi Inoue, and Til Schuermann, I examine the common practice of converting one-day volatility estimates to "h-day" estimates by scaling by the square root of h. This turns out to be inappropriate except under very special circumstances routinely violated in practice. [16] Another more broadly focused study with Christoffersen uses a model-free procedure to assess the forecastability of volatility at various horizons ranging from a day to a month. [17] Perhaps surprisingly, the forecastability of volatility turns out to decay rather quickly with the horizon. Th is suggests that volatility forecastability, although clearly relevant for risk management at short horizons, may be much less important at longer horizons. We are currently at an interesting juncture in regard to long-horizon volatility forecastability: some studies are indicating long memory in volatility forecastability and others are not. Very much related is the possibility of structural breaks, which can masquerade as long memory. This is an important direction for future research, and I have begun to tackle it in recent work with Inoue. [18]

Econometric Methods for Business Cycle and Macroeconomic Modeling

After nearly a decade of strong growth, it is tempting to assert that the business cycle is dead. It is not. Indeed, a recession is coming -- we just don't know when. Another strand of my work, much of it with Glenn D. Rudebusch, centers on the econometrics of business cycles and business cycle modeling. In part, the research is eclectic and scattered, ranging from early work on business cycle duration dependence to later work on strategic complementarity and job durations. [19] But much of it is organized around three general themes, which I discuss briefly in turn. [20]

What are the defining characteristics of the business cycle? Two features are crucial. The first involves the co-movement of economic variables over the cycle, or, roughly speaking, how broadly business cycles are spread throughout the economy. The notion of co-movement -- particularly accelerated or delayed co-movement -- leads naturally to notions of coincident, leading, and lagging business cycle indicators. The second feature involves the timing of the slow switching between expansions and contractions, and the persistence of business cycle regimes.

Central to much of the work is the idea of a dynamic factor model with a Markov switching factor, which simultaneously captures both co-movement and regime switching, [21] as recently implemented using Markov chain Monte Carlo methods. [22]

How can business cycle models be evaluated? One way or another, we want to assess business cycle models empirically, by checking whether the properties of our model economy match those of the real economy. However, doing so in a rigorous fashion presents challenges, particularly with the modern breed of dynamic stochastic general equilibrium models. In recent work with Lee E. Ohanian, I have attempted to provide a constructive framework for assessing agreement between dynamic equilibrium models and data, which enables a complete comparison of model and data means, variances, and serial correlations. [23] The new methods use bootstrap algorithms to evaluate the significance of deviations between model and data without assuming that the model under investigation is correctly specified. They also use goodness-of-fit criteria to produce estimators that optimize economically relevant loss functions.

In related work, Lutz Kilian and I propose a measure of predictability based on the ratio of the expected loss of a short-run forecast to the expected loss of a long-run forecast. [24] The predictability measure can be tailored to the forecast horizons of interest, and it allows for general loss functions, univariate or multivariate information sets, and stationary or nonstationary data. We propose a simple estimator, and we suggest resampling methods for inference. We then put the new tools to work in macroeconomic environments. First, based on fitted parametric models, we assess the predictability of a variety of macroeconomic series. Second, we analyze the internal propagation mechanism of a standard dynamic macroeconomic model by comparing the predictability of model inputs and model outputs. Finally, we compare the predictability in U.S. macroeconomic data with that implied by leading macroeconomic models.

How can secular growth be distinguished from cyclical fluctuations? Understanding the difference between the economy's trend and its cycle is crucial for business cycle analysis. A long debate continues on the appropriate separation of trend and cycle; Abdelhak S. Senhadji and I have summarized recent elements in this debate and attempted to sift the relevant evidence. [25] In the end, a great deal of uncertainty remains; however, it appears that some traditional trend/cycle decompositions with quite steady trend growth are not bad approximations in practice.

If there is still uncertainty in disentangling trend from cycle, there is less in finding good cyclical forecasting models. In particular, the low power that plagues unit root tests and related procedures when testing against nearby alternatives, which are typically the relevant alternatives in macroeconomics and finance, is not necessarily a concern for forecasting. Ultimately, the question of interest for forecasting is not whether unit root pretests select the "true" model, but whether they select models that produce superior forecasts. My recent work with Kilian suggests that unit root tests are effective when used for that purpose. [26]

(*.)Diebold is a Research Associate in the NBER's Program on Asset Pricing, Armellino Visiting Professor of Finance at New York University's Stern School of Business, and Lawrence R. Klein Professor of Economics and Statistics and Director of the Institute for Economic Research at the University of Pennsylvania. His "Profile" appears later in this issue.

(1.) J. H. Cochrane, "New Facts in Finance," NBER Working Paper No. 7169, June 1999, provides a fine survey of many of the recent developments in finance.

(2.) The "Forecasting and Empirical Methods in Finance and Macroeconomics" group is supported by the NBER and the National Science Foundation. Its meetings have produced several associated symposiums; including those whose proceedings appear in: Review of Economics and Statistics, November 1999, F X. Diebold; J. H. Stock, and K. D. West; eds.; International Economic Review, November 1998, F. X. Diebold and K. D. West, eds.; and Journal of Applied Econometrics, September--October 1996, F. X. Diebold and M. W. Watson, eds.

(3.) For overviews, see F. X. Diebold, Elements of Forecasting, Cincinnati: SouthWestern College Publishing, 1998, and "The Pat, Present, and Future of Macroeconomic Forecasting," NBER Working Paper No. 6290, November 1997, and Journal of Economic Perspectives, 12 (1998), pp. 175--92.

(4.) See, for example, A. C. Stockman, "Economic Theory and Exchange Rate Forecasts," International Journal of Forecasting, 3 (1987), pp. 3--15.

(5.) P. F. Christoffersen and F. X. Diebold, "Optimal Prediction under Asymmetric Loss," NBER Technical Working Paper No. 167, October 1994. Published in two parts, as "Optimal Prediction under Asymmetric Loss," Econometric Theory, 13 (1997), pp. 808--17, and "Further Results on Forecasting and Model Selection under Asymmetric Loss," Journal of Applied Econometrics, 11 (1996), pp. 561-72.

(6.) P. F. Christoffersen and F. X. Diebold, "Cointegration and Long-Horizon Forecasting," NBER Technical Working Paper No. 217, October 1997, and Journal of Business and Economic Statistics, 16 (1998), pp. 450--8.

(7.) For a survey, see F. X. Diebold and J. A. Lopez, "Forecast Evaluation and Combination," NBER Technical Working Paper No. 192, March 1996, and Handbook of Statistics, G. S. Maddala and C. R. Rao, eds., pp. 241--68. Amsterdam: NorthHolland, 1996.

(8.) F. X. Diebold and R. S. Mariano, "Comparing Predictive Accuracy," Journal of Business and Economic Statistics, 13 (1995), pp. 253--65, and in Economic Forecasting, T. C. Mills, ed., Cheltenham, U.K.: Edward Elgar Publishing, 1998.

(9.) K. D. West, "Asymptotic Inference about Predictive Ability," Econometrica, 64 (1996), pp. 1067--84.

(10.) H. White, "A Reality Check for Data Snooping," Econometrica, forthcoming, and R. Sullivan, A. Timmermann, and H. White; "Data Snooping, Technical Trading Rule Performance, and the Bootstrap," Journal of Finance, forthcoming.

(11.) F. X. Diebold, T. A. Gunther, and A. S. Tay, "Evaluating Density Forecasts, with Applications to Financial Risk Management," NBER Technical Working Paper No. 215, October 1997, and International Economic Review, 39 (1998), pp. 863--83.

(12.) F. X. Diebold, J. Hahn, and A. S. Tay, "Multivariate Density Forecast Evaluation and Calibration in Financial Risk Management: High-Frequency Returns on Foreign Exchange," revised and re-titled version of NBER Working Paper No. 6845, December 1998. Forthcoming in Review of Economics and Statistics, 81 (1999).

(13.) F. X. Diebold, A. S. Tay, and K. F Wallis, "Evaluating Density Forecasts of Inflation: The Survey of Professional Forecasters," NBER Working Paper No. 6228, October 1997, and Cointegration, Causality, and Forecasting: A Festschrift in Honor of Clive W. J. Granger, R. Engle and H. White, eds., pp. 76-90. Oxford: Oxford University Press, 1999.

(14.) For an overview, see T. Andersen, T. Bollerslev, F. X. Diehold, and P. Labys, "Understanding, Optimizing, Using, and Forecasting Realized Volatility and Correlation," Risk, 12 (forthcoming).

(15.) See T. Andersen, T. Bollerslev, F. X. Diebold, and P. Labys, "The Distribution of Exchange Rate Volatility," NBER Working Paper No. 6961, February 1999, and the references therein.

(16.) See F. X. Diehold, A. Hickman, A. Inoue, and T. Schuermann, "Scale Models," Risk, 11 (1998), pp. 104--7, and Hedging with Trees: Advances in Pricing and Risk Managing Derivatives, M. Broadie and P. Glasserman, eds., pp. 233--7. London: Risk Publications, 1998.

(17.) P. F. Christoffersen and F. X. Diebold, "How Relevant Is Volatility Forecasting for Financial Risk Management?" NBER Working Paper No. 6844, December 1998, and Review of Economics and Statistics, 82 (forthcoming).

(18.) F. X. Diebold and A. Inoue, "Long Memory and Structural Change," manuscript, Department of Finance, Stern School of Business, New York University, 1999.

(19.) F. X. Diebold and G. D. Rudebusch, "A Nonparametric Investigation of Duration Dependence in the American Business Cycle," Journal of Political Economy, 98 (1990), pp. 596--616; F. X. Diehold, D. Neumark and D. Polsky, "Job Stability in the United States," Journal of Labor Economics, 15 (1997), pp. 206--33; and A. N. Bomfim and F. X. Diebold, "Bounded Rationality and Strategic Complementarity in a Macroeconomic Model: Policy Effects, Persistence, and Multipliers," Economic Journal, 107 (1997), pp. 1358--75.

(20.) F. X. Diehold and G. D. Rudebusch, Business Cycles: Durations, Dynamics, and Forecasting. Princeton: Princeton University Press, 1999.

(21.) F. X. Diebold and C. D. Rudebusch, "Measuring Business Cycles: A Modern Perspective," Review of Economics and Statistics, 78 (1996), pp. 67--77; and F. X. Diehold, J.-H. Lee, and G. Weinbach, "Regime Switching with Time-Varying Transition Probabilities," in Nonstationary Time Series Analysis and Cointegration, C. Hargreaves, ed., pp. 283--302. Oxford: Oxford University Press, 1994.

(22.) C-J. Kim and C. R. Nelson, "Business Cycle Turning Points, a New Coincident Index, and Tests of Duration Dependence Based on a Dynamic Factor Model with Regime-Switching," Review of Economics and Statistics, 80 (1998), pp. 188--201, and State Space Models with Regime Switching. Cambridge, Mass.: MIT Press; 1999.

(23.) F. X. Diebold, L. E. Ohanian, and J. Berkowitz, "Dynamic Equilibrium Economies. A Framework for Comparing Models and Data," NBER Technical Working Paper No. 174, February 1995, and Review of Economic Studies, 65 (1998), pp. 433--52.

(24.) F. X. Diebold and L. Kilian, "Measuring Predictability: Theory and Macroeconomic Applications," NBER Technical Working Paper No. 213, August 1997.

(25.) F. X. Diebold and A. S. Senhadji, "The Uncertain Unit Root in Real GNP: Comment;" American Economic Review, 86 (1996), pp. 1291--8.

(26.) F. X. Diebold and L. Kilian "Unit Root Tests Are Useful for Selecting Forecasting Models," NBER Working Paper No. 6928, February 1999, and forthcoming in Journal of Business and Economic Statistics, 18.