MMSE Forecasting of Conditional Mean Models

What Are MMSE Forecasts?

A common objective of time series modeling is generating forecasts for a process over a future time horizon. That is, given an observed series y₁, y₂,...,y_N and a forecast horizon h, generate predictions for $y_{N + 1}, y_{N + 2}, \dots, y_{N + h} .$

Let ${\hat{y}}_{t + 1}$ denote a forecast for the process at time t + 1, conditional on the history of the process up to time t, H_t, and the exogenous covariate series up to time t + 1, X_{t + 1}, if a regression component is included in the model. The minimum mean square error (MMSE) forecast is the forecast ${\hat{y}}_{t + 1}$ that minimizes expected square loss,

$E {(y_{t + 1} - {\hat{y}}_{t + 1} | H_{t}, X_{t + 1})}^{2} .$

Minimizing this loss function yields the MMSE forecast,

${\hat{y}}_{t + 1} = E (y_{t + 1} | H_{t}, X_{t + 1}) .$

How `forecast` Generates MMSE Forecasts

The forecast function generates MMSE forecasts recursively. When you call forecast, you specify the model Mdl, forecast horizon numperiods, and presample responses Y0. You can optionally specify the presample innovations 'E0', conditional variances 'V0', and exogenous data 'X0' by using name-value pair arguments. Although forecast does not require X0 or forecast sample exogenous data XF, if you specify X0 then you must also specify XF.

To begin forecasting from the end of an observed series, say Y, use the last few observations of Y as presample responses Y0 to initialize the forecast. There are several points to keep in mind when you specify presample data:

The minimum number of responses needed to initialize forecasting is stored in the property P of an arima model. If you provide too few presample observations, forecast returns an error.
If you forecast a model with an MA component, then forecast requires presample innovations. The number of innovations needed is stored in the property Q of an arima model. If you also have a conditional variance model, you must additionally account for any presample innovations it requires. If you specify presample innovations, but not enough, forecast returns an error.
If you do not specify any presample innovations, but specify sufficient presample responses (at least P + Q) and exogenous covariate data (at least the number of presample responses minus P), then forecast automatically infers presample innovations. In general, the longer the presample response series you provide, the better the inferred presample innovations will be. If you provide presample responses and exogenous covariate data, but not enough, forecast sets presample innovations equal to zero.
If you forecast a model with a regression component, then forecast requires future exogenous covariate data for all time points in the forecast period (numperiods). If you provide future exogenous covariate data, but not enough, then forecast returns an error.

Consider generating forecasts for an AR(2) process,

$y_{t} = c + ϕ_{1} y_{t - 1} + ϕ_{2} y_{t - 2} + ε_{t} .$

Given presample observations $y_{N - 1}$ and $y_{N},$ forecasts are recursively generated as follows:

${\hat{y}}_{N + 1} = c + ϕ_{1} y_{N} + ϕ_{2} y_{N - 1}$
${\hat{y}}_{N + 2} = c + ϕ_{1} {\hat{y}}_{N + 1} + ϕ_{2} y_{N}$
${\hat{y}}_{N + 3} = c + ϕ_{1} {\hat{y}}_{N + 2} + ϕ_{2} {\hat{y}}_{N + 1}$

$⋮$

For a stationary AR process, this recursion converges to the unconditional mean of the process,

$μ = \frac{c}{(1 - ϕ_{1} - ϕ_{2})} .$

For an MA(2) process, e.g.,

$y_{t} = μ + ε_{t} + θ_{1} ε_{t - 1} + θ_{2} ε_{t - 2},$

you need 2 presample innovations to initialize the forecasts. All innovations from time N + 1 and greater are set to their expectation, zero. Thus, for an MA(2) process, the forecast for any time more than 2 steps in the future is the unconditional mean, μ.

Forecast Error

The forecast mean square error for an s-step ahead forecast is given by

$MSE = E {(y_{t + s} - {\hat{y}}_{t + s} | H_{t + s - 1}, X_{t + s})}^{2} .$

Consider a conditional mean model given by

$y_{t} = μ + x_{t}^{'} β + ψ (L) ε_{t},$

where $ψ (L) = 1 + ψ_{1} L + ψ_{2} L^{2} + \dots$ . Sum the variances of the lagged innovations to get the s-step MSE,

$(1 + ψ_{1}^{2} + ψ_{2}^{2} + \dots + ψ_{s - 1}^{2}) σ_{ε}^{2},$

where $σ_{ε}^{2}$ denotes the innovation variance.

For stationary processes, the coefficients of the infinite lag operator polynomial are absolutely summable, and the MSE converges to the unconditional variance of the process.

For nonstationary processes, the series does not converge, and the forecast error grows over time.