Conduct Lagrange Multiplier Test

Open Live Script

This example shows how to calculate the required inputs for conducting a Lagrange multiplier (LM) test with lmtest. The LM test compares the fit of a restricted model against an unrestricted model by testing whether the gradient of the loglikelihood function of the unrestricted model, evaluated at the restricted maximum likelihood estimates (MLEs), is significantly different from zero.

The required inputs for lmtest are the score function and an estimate of the unrestricted variance-covariance matrix evaluated at the restricted MLEs. This example compares the fit of an AR(1) model against an AR(2) model.

Compute Restricted MLE

Obtain the restricted MLE by fitting an AR(1) model (with a Gaussian innovation distribution) to the given data. Assume you have presample observations ( $y_{- 1}$ , $y_{0}$ ) = (9.6249,9.6396).

Y = [10.1591; 10.1675; 10.1957; 10.6558; 10.2243; 10.4429;
     10.5965; 10.3848; 10.3972;  9.9478;  9.6402;  9.7761;
     10.0357; 10.8202; 10.3668; 10.3980; 10.2892;  9.6310;
      9.6318;  9.1378;  9.6318;  9.1378];
Y0 = [9.6249; 9.6396];

Mdl = arima(1,0,0);
EstMdl = estimate(Mdl,Y,'Y0',Y0);

 
    ARIMA(1,0,0) Model (Gaussian Distribution):
 
                 Value     StandardError    TStatistic     PValue  
                _______    _____________    __________    _________

    Constant     3.2999        2.4606         1.3411        0.17988
    AR{1}       0.67097       0.24635         2.7237      0.0064564
    Variance    0.12506      0.043015         2.9074      0.0036441

When conducting an LM test, only the restricted model needs to be fit.

Compute Gradient Matrix

Estimate the variance-covariance matrix for the unrestricted AR(2) model using the outer product of gradients (OPG) method.

For an AR(2) model with Gaussian innovations, the contribution to the loglikelihood function at time $t$ is given by

$\log L_{t} = - 0.5 \log (2 π σ_{ε}^{2}) - \frac{(y_{t} - c - ϕ_{1} y_{t - 1} - ϕ_{2} y_{t - 2})^{2}}{2 σ_{ε}^{2}}$

where $σ_{ε}^{2}$ is the variance of the innovation distribution.

The contribution to the gradient at time $t$ is

$[\begin{array}{c} \frac{\partial \log L_{t}}{\partial c} & \frac{\partial \log L_{t}}{\partial ϕ_{1}} & \frac{\partial \log L_{t}}{\partial ϕ_{2}} & \frac{\partial \log L_{t}}{\partial σ_{ε}^{2}} \end{array}],$

where

$\begin{array}{c} \frac{\partial \log L_{t}}{\partial c} & = & \frac{y_{t} - c - ϕ_{1} y_{t - 1} - ϕ_{2} y_{t - 2}}{σ_{ε}^{2}} \\ \frac{\partial \log L_{t}}{\partial ϕ_{1}} & = & \frac{y_{t - 1} (y_{t} - c - ϕ_{1} y_{t - 1} - ϕ_{2} y_{t - 2})}{σ_{ε}^{2}} \\ \frac{\partial \log L_{t}}{\partial ϕ_{2}} & = & \frac{y_{t - 2} (y_{t} - c - ϕ_{1} y_{t - 1} - ϕ_{2} y_{t - 2})}{σ_{ε}^{2}} \\ \frac{\partial \log L_{t}}{\partial σ_{ε}^{2}} & = & - \frac{1}{2 σ_{ε}^{2}} + \frac{(y_{t} - c - ϕ_{1} y_{t - 1} - ϕ_{2} y_{t - 2})^{2}}{2 σ_{ε}^{4}} \end{array}$

Evaluate the gradient matrix, $G$ , at the restricted MLEs (using ${ϕ_{}^{ˆ}}_{2} = 0$ ).

c = EstMdl.Constant;
phi1 = EstMdl.AR{1};
phi2 = 0;
sig2 = EstMdl.Variance;

Yt = Y;
Yt1 = [9.6396; Y(1:end-1)];
Yt2 = [9.6249; Yt1(1:end-1)];

N = length(Y);
G = zeros(N,4);
G(:,1) = (Yt-c-phi1*Yt1-phi2*Yt2)/sig2;
G(:,2) = Yt1.*(Yt-c-phi1*Yt1-phi2*Yt2)/sig2;
G(:,3) = Yt2.*(Yt-c-phi1*Yt1-phi2*Yt2)/sig2;
G(:,4) = -0.5/sig2 + 0.5*(Yt-c-phi1*Yt1-phi2*Yt2).^2/sig2^2;

Estimate Variance-Covariance Matrix

Compute the OPG variance-covariance matrix estimate.

V = inv(G'*G)

V = 4×4

    6.1431   -0.6966    0.0827    0.0367
   -0.6966    0.1535   -0.0846   -0.0061
    0.0827   -0.0846    0.0771    0.0024
    0.0367   -0.0061    0.0024    0.0019

Numerical inaccuracies can occur due to computer precision. To make the variance-covariance matrix symmetric, combine half of its value with half of its transpose.

V = V/2 + V'/2;

Calculate Score Function

Evaluate the score function (the sum of the individual contributions to the gradient).

score = sum(G);