Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast

FORECASTING AND MODEL SELECTION
Anurag Prasad
Department of Mathematics and Statistics Indian Institute of Technology Kanpur, India

REACH Symposium, March 15-18, 2008

1

Forecasting and Model Selection

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast

Outline

1

Modeling and Forecasting

2

Forecasting Methods

3

Measuring the Accuracy of Forecast

2

Forecasting and Model Selection

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Assumptions of Forecasting 1 Element of Uncertainty Blind Spots Change in Forecast Accuracy 2 3 3 Forecasting and Model Selection .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Framework of a Forecast System Model−Building Phase Theory and/or Historical Data Forecasting Phase Model Specification Model Estimation No Diagnostic Checking Yes No Forecast Generation Stability Checking Yes New Observations Forecast Updation Data 4 Forecasting and Model Selection .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Choice of a Particular Forecast Model 1 Degree of Accuracy Required Cost of Producing Forecasts Forecast Horizon Degree of Complexity Required Available Data 2 3 4 5 5 Forecasting and Model Selection .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Classification of Estimation Methods 1 Time Series Methods Causal Methods Judgemental Methods 2 3 6 Forecasting and Model Selection .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Time Series Methods • Use historical data as a basis • Underlying patterns are fairly stable 1 2 3 4 5 6 7 Autoregressive Moving Average (ARMA) Exponential Smoothing Extrapolation Linear Prediction Trend Estimation Growth Curve Box-Jenkins Approach 7 Forecasting and Model Selection .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Causal Methods • Belief that some other time series can be useful • Assumption that it is possible to identify the underlying factors 1 Regression Analysis * Linear Regression * Non-Linear Regression Econometrics 2 8 Forecasting and Model Selection .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Judgemental Methods • Incorporate intuitive judgements. opinions and probability estimates 1 2 3 4 5 6 Composite Forecasts Surveys Delphi Method Scenario Building Technology Forecasting Forecast by Analogy 9 Forecasting and Model Selection .

. y(t) : Forecast value at period t. e(t) = y (t) − y (t) y(t) ^ ) y(t i y(t ) i t t i 10 Forecasting and Model Selection . . N. y (t) : Actual value at period t. e(t) : Forecast error at period t.Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Forecast Error For t = 1. . .

11 Forecasting and Model Selection . • Departure of points from the 450 line through origin indicates imperfect forecasts.Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Graphical Measures of Forecast Accuracy Plot of y (t) versus y (t) • Keep the same scale for both the axes.

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast y(t) y(t) ^ y(t) Correct Model Form ^ y(t) Incorrect Model Form 12 Forecasting and Model Selection .

13 Forecasting and Model Selection . the forecast errors should vary in a horizontal band around zero. • For a good model.Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Plot of e(t) versus t • Reveals patterns of variability which the model has failed to explain.

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast e(t) 0 e(t) 0 t Correct Model Form t Incorrect Model Form e(t) 0 e(t) 0 t Incorrect Model Form t Incorrect Model Form 14 Forecasting and Model Selection .

.Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Descriptive Measures of Forecast Accuracy Descriptive Measures of Forecast Accuracy are used to. easily interpreted measure of model’s reliability Compare the accuracy of two different models Search for an optimal model Monitor a model’s performance 2 3 4 15 Forecasting and Model Selection .. 1 Provide a single.

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Descriptive Measures of Forecast Accuracy Mean Absolute Deviation (MAD) MAD = |forecast error| number of forecasts = N t=1   |e(t)| N Mean Square Error (MSE) MSE = =   (forecast error)2 number of forecasts   Root Mean Square Error (RMSE) √ RMSE = MSE   N t=1 e(t)2 N 16 Forecasting and Model Selection .

100% N Pearson’s Correlation Coefficient (r ) between y (t) and y (t) ¡ ¡   ¢ r= N ¯ ¯ t=1 (y (t)−y )(y (t)−y) N N ¯ 2 ¯ 2 t=1 (y (t)−y ) t=1 (y (t)−y) ¡ ¡   17 Forecasting and Model Selection .Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Descriptive Measures of Forecast Accuracy Mean Absolute Percent Error (MAPE) MAPE = = N t=1       ¢ |forecast error/actual value| .100% number of forecasts |e(t)/y (t)| .

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Descriptive Measures of Forecast Accuracy "No Change" model is : y (t + 1) = y (t) Theil’s Inequality Coefficient (U) U= RMSE("new" model) RMSE("no change" model) U>1 U=1 U<1 ⇒ ⇒ ⇒ worse than "no change" model as good as "no change" model better than "no change" model 18 Forecasting and Model Selection .

Statistical Methods for Forecasting. P. Ledolter. Abraham and J. John Wiley & Sons Introduction to Time Series and Forecasting.J. 1989. Farnum and L. Academic Press 19 Forecasting and Model Selection . Springer-Verlag Time Series Analysis and Forecasting. B. Davis. Brockwell and R.A. PWS-KENT Publishing Co. N.Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Further Readings Quantitative Forecasting Methods. Stanton. R. 1983.W. 2000.R. 2002 (Second Edition) . Yaffee.

Modeling and Forecasting Forecasting Methods Measuring the Accuracy of Forecast Thank You 20 Forecasting and Model Selection .

Sign up to vote on this title
UsefulNot useful