Main Content

step

Improve generalized linear regression model by adding or removing terms

collapse all in page

Syntax

NewMdl = step(mdl)
NewMdl = step(mdl,Name,Value)

Description

NewMdl = step(mdl) returns a generalized linear regression model based on mdl using stepwise regression to add or remove one predictor.

NewMdl = step(mdl,Name,Value) specifies additional options using one or more name-value pair arguments. For example, you can specify the criterion to use to add or remove terms and the maximum number of steps to take.

example

Examples

collapse all

Open Live Script

Fit a Poisson regression model using random data and a single predictor, and then use step to improve the model by adding or removing predictor terms.

Generate sample data that has 20 predictor variables. Use three of the predictors to generate the Poisson response variable. 

rng('default') % For reproducibility
X = randn(100,20);
mu = exp(X(:,[5 10 15])*[.4;.2;.3] + 1);
y = poissrnd(mu);

Create a generalized linear regression model of Poisson data using X(:,2) as the only predictor. 

mdl = fitglm(X,y,'y ~ x2','Distribution','poisson')
mdl = 
Generalized linear regression model:
    log(y) ~ 1 + x2
    Distribution = Poisson

Estimated Coefficients:
                   Estimate       SE        tStat       pValue  
                   ________    ________    _______    __________

    (Intercept)      1.1386    0.056722     20.073    1.2817e-89
    x2             0.010768    0.056564    0.19037       0.84902


100 observations, 98 error degrees of freedom
Dispersion: 1
Chi^2-statistic vs. constant model: 0.0362, p-value = 0.849

Improve mdl by using step. Specify 'NSteps' as 5 to allow at most 5 steps of stepwise regression.

mdl1 = step(mdl,'NSteps',5)
1. Adding x5, Deviance = 134.4375, Chi2Stat = 52.21338, PValue = 4.978574e-13
2. Adding x15, Deviance = 106.1925, Chi2Stat = 28.24496, PValue = 1.068927e-07
3. Adding x10, Deviance = 94.708, Chi2Stat = 11.4845, PValue = 0.000701792
4. Removing x2, Deviance = 95.021, Chi2Stat = 0.31263, PValue = 0.57607

mdl1 = 
Generalized linear regression model:
    log(y) ~ 1 + x5 + x10 + x15
    Distribution = Poisson

Estimated Coefficients:
                   Estimate       SE       tStat       pValue  
                   ________    ________    ______    __________

    (Intercept)     1.0115     0.064275    15.737    8.4217e-56
    x5             0.39508     0.066665    5.9263    3.0977e-09
    x10            0.18863      0.05534    3.4085     0.0006532
    x15            0.29295     0.053269    5.4995    3.8089e-08


100 observations, 96 error degrees of freedom
Dispersion: 1
Chi^2-statistic vs. constant model: 91.7, p-value = 9.61e-20

step adds the three predictor variables used to generate the response variable to the model and removes X(:,2) from the model.

Input Arguments

collapse all

Generalized linear regression model, specified as a GeneralizedLinearModel object created using fitglm or stepwiseglm.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: 'Criterion','aic','Upper','quadratic','Verbose',2 instructs step to use the Akaike information criterion, include (at most) the quadratic terms in the model, and display the evaluation process and the decision taken at each step.

Criterion to add or remove terms, specified as the comma-separated pair consisting of 'Criterion' and one of these values:

  • 'Deviance' — p-value for an F-test or chi-squared test of the change in the deviance that results from adding or removing the term. The F-test tests a single model, and the chi-squared test compares two different models.

  • 'sse' — p-value for an F-test of the change in the sum of squared error that results from adding or removing the term.

  • 'aic' — Change in the value of the Akaike information criterion (AIC).

  • 'bic' — Change in the value of the Bayesian information criterion (BIC).

  • 'rsquared' — Increase in the value of R2.

  • 'adjrsquared' — Increase in the value of adjusted R2.

Example: 'Criterion','bic'

Model specification describing terms that cannot be removed from the model, specified as the comma-separated pair consisting of 'Lower' and one of these values:

  • A character vector or string scalar naming the model.

    ValueModel Type
    'constant'Model contains only a constant (intercept) term.
    'linear'Model contains an intercept and linear term for each predictor.
    'interactions'Model contains an intercept, linear term for each predictor, and all products of pairs of distinct predictors (no squared terms).
    'purequadratic'Model contains an intercept term and linear and squared terms for each predictor.
    'quadratic'Model contains an intercept term, linear and squared terms for each predictor, and all products of pairs of distinct predictors.
    'polyijk'Model is a polynomial with all terms up to degree i in the first predictor, degree j in the second predictor, and so on. Specify the maximum degree for each predictor by using numerals 0 though 9. The model contains interaction terms, but the degree of each interaction term does not exceed the maximum value of the specified degrees. For example, 'poly13' has an intercept and x1, x2, x22, x23, x1*x2, and x1*x22 terms, where x1 and x2 are the first and second predictors, respectively.

  • A t-by-(p + 1) matrix, or a Terms Matrix, specifying terms in the model, where t is the number of terms, p is the number of predictor variables, and +1 accounts for the response variable. A terms matrix is convenient when the number of predictors is large and you want to generate the terms programmatically.

  • A character vector or string scalar Formula in the form

    'Y ~ terms',

    where the terms are in Wilkinson Notation. The variable names in the formula must be valid MATLAB® identifiers.

Example: 'Lower','linear'

Data Types: char | string | single | double

Maximum number of steps to take, specified as the comma-separated pair consisting of 'NSteps' and a positive integer.

Example: 'NSteps',5

Data Types: single | double

Threshold for the criterion to add a term, specified as a scalar value described in this table.

CriterionDefault ValueDecision
'Deviance'0.05If the p-value of the F-statistic or chi-squared statistic is less than PEnter (p-value to enter), add the term to the model.
'SSE'0.05If the p-value of the F-statistic is less than PEnter, add the term to the model.
'AIC'0If the change in the AIC of the model is less than PEnter, add the term to the model.
'BIC'0If the change in the BIC of the model is less than PEnter, add the term to the model.
'Rsquared'0.1If the increase in the R-squared value of the model is greater than PEnter, add the term to the model.
'AdjRsquared'0If the increase in the adjusted R-squared value of the model is greater than PEnter, add the term to the model.

To prevent forward stepwise regression, specify PEnter as a value that step cannot achieve.

  • When Criterion is "Deviance", "SSE", "AIC", or "BIC", specify PEnter as –Inf.

  • When Criterion is "Rsquared" or "AdjRsquared", specify PEnter as Inf.

Example: PEnter=0.075

Threshold for the criterion to remove a term, specified as a scalar value, described in this table.

CriterionDefault ValueDecision
'Deviance'0.10If the p-value of the F-statistic or chi-squared statistic is greater than PRemove (p-value to remove), remove the term from the model.
'SSE'0.10If the p-value of the F-statistic is greater than PRemove, remove the term from the model.
'AIC'0.01If the change in the AIC of the model is greater than PRemove, remove the term from the model.
'BIC'0.01If the change in the BIC of the model is greater than PRemove, remove the term from the model.
'Rsquared'0.05If the increase in the R-squared value of the model is less than PRemove, remove the term from the model.
'AdjRsquared'-0.05If the increase in the adjusted R-squared value of the model is less than PRemove, remove the term from the model.

At each step, the step function also checks whether a term is redundant (linearly dependent) with other terms in the current model. When a term is linearly dependent on other terms in the current model, the step function removes the redundant term, regardless of the criterion value. For more information, see Criterion.

To prevent backward stepwise regression, specify PRemove as a value that step cannot achieve.

  • When Criterion is "Deviance", "SSE", "AIC", or "BIC", specify PRemove as Inf.

  • When Criterion is "Rsquared" or "AdjRsquared", specify PRemove as –Inf.

Example: PRemove=0.05

Model specification describing the largest set of terms in the fit, specified as the comma-separated pair consisting of 'Upper' and one of these values:

  • A character vector or string scalar naming the model.

    ValueModel Type
    'constant'Model contains only a constant (intercept) term.
    'linear'Model contains an intercept and linear term for each predictor.
    'interactions'Model contains an intercept, linear term for each predictor, and all products of pairs of distinct predictors (no squared terms).
    'purequadratic'Model contains an intercept term and linear and squared terms for each predictor.
    'quadratic'Model contains an intercept term, linear and squared terms for each predictor, and all products of pairs of distinct predictors.
    'polyijk'Model is a polynomial with all terms up to degree i in the first predictor, degree j in the second predictor, and so on. Specify the maximum degree for each predictor by using numerals 0 though 9. The model contains interaction terms, but the degree of each interaction term does not exceed the maximum value of the specified degrees. For example, 'poly13' has an intercept and x1, x2, x22, x23, x1*x2, and x1*x22 terms, where x1 and x2 are the first and second predictors, respectively.

  • A t-by-(p + 1) matrix, or a Terms Matrix, specifying terms in the model, where t is the number of terms, p is the number of predictor variables, and +1 accounts for the response variable. A terms matrix is convenient when the number of predictors is large and you want to generate the terms programmatically.

  • A character vector or string scalar Formula in the form

    'Y ~ terms',

    where the terms are in Wilkinson Notation. The variable names in the formula must be valid MATLAB identifiers.

Example: 'Upper','quadratic'

Data Types: char | string | single | double

Control for the display of information, specified as the comma-separated pair consisting of 'Verbose' and one of these values:

  • 0 — Suppress all display.

  • 1 — Display the action taken at each step.

  • 2 — Display the evaluation process and the action taken at each step.

Example: 'Verbose',2

Output Arguments

collapse all

Generalized linear regression model, returned as a GeneralizedLinearModel object.

To overwrite the input argument mdl, assign the new model to mdl.

mdl = step(mdl);

More About

collapse all

Terms Matrix

A terms matrix T is a t-by-(p + 1) matrix specifying terms in a model, where t is the number of terms, p is the number of predictor variables, and +1 accounts for the response variable. The value of T(i,j) is the exponent of variable j in term i.

For example, suppose that an input includes three predictor variables x1, x2, and x3 and the response variable y in the order x1, x2, x3, and y. Each row of T represents one term:

  • [0 0 0 0] — Constant term or intercept

  • [0 1 0 0] — x2; equivalently, x1^0 * x2^1 * x3^0

  • [1 0 1 0] — x1*x3

  • [2 0 0 0] — x1^2

  • [0 1 2 0] — x2*(x3^2)

The 0 at the end of each term represents the response variable. In general, a column vector of zeros in a terms matrix represents the position of the response variable. If you have the predictor and response variables in a matrix and column vector, then you must include 0 for the response variable in the last column of each row.

Formula

A formula for model specification is a character vector or string scalar of the form 'y ~ terms'.

  • y is the response name.

  • terms represents the predictor terms in a model using Wilkinson notation.

To represent predictor and response variables, use the variable names of the table input tbl or the variable names specified by using VarNames. The default value of VarNames is {'x1','x2',...,'xn','y'}.

For example:

  • 'y ~ x1 + x2 + x3' specifies a three-variable linear model with intercept.

  • 'y ~ x1 + x2 + x3 – 1' specifies a three-variable linear model without intercept. Note that formulas include a constant (intercept) term by default. To exclude a constant term from the model, you must include –1 in the formula.

A formula includes a constant term unless you explicitly remove the term using –1.

Wilkinson Notation

Wilkinson notation describes the terms in a model. The notation relates to the terms included in the model, not to the multipliers (coefficients) of those terms.

Wilkinson notation uses these symbols:

  • + means include the next variable.

  • – means do not include the next variable.

  • : defines an interaction, which is a product of the terms.

  • * defines an interaction and all lower order terms.

  • ^ raises the predictor to a power, exactly as in * repeated, so ^ includes lower order terms as well.

  • () groups the terms.

This table shows typical examples of Wilkinson notation.

Wilkinson NotationTerms in Standard Notation
1Constant (intercept) term
x1^k, where k is a positive integerx1, x12, ..., x1k
x1 + x2x1, x2
x1*x2x1, x2, x1*x2
x1:x2x1*x2 only
–x2Do not include x2
x1*x2 + x3x1, x2, x3, x1*x2
x1 + x2 + x3 + x1:x2x1, x2, x3, x1*x2
x1*x2*x3 – x1:x2:x3x1, x2, x3, x1*x2, x1*x3, x2*x3
x1*(x2 + x3)x1, x2, x3, x1*x2, x1*x3

For more details, see Wilkinson Notation.

Algorithms

  • Stepwise regression is a systematic method for adding and removing terms from a linear or generalized linear model based on their statistical significance in explaining the response variable. The method begins with an initial model, specified using modelspec, and then compares the explanatory power of incrementally larger and smaller models.

    The step function uses forward and backward stepwise regression to determine a final model. At each step, the function searches for terms to add to the model or remove from the model based on the value of the 'Criterion' name-value pair argument.

    The default value of 'Criterion' for a linear regression model is 'sse'. In this case, stepwiselm and step of LinearModel use the p-value of an F-statistic to test models with and without a potential term at each step. If a term is not currently in the model, the null hypothesis is that the term would have a zero coefficient if added to the model. If there is sufficient evidence to reject the null hypothesis, the function adds the term to the model. Conversely, if a term is currently in the model, the null hypothesis is that the term has a zero coefficient. If there is insufficient evidence to reject the null hypothesis, the function removes the term from the model.

    Stepwise regression takes these steps when 'Criterion' is 'sse':

    1. Fit the initial model.

    2. Examine a set of available terms not in the model. If any of the terms have p-values less than an entrance tolerance (that is, if it is unlikely a term would have a zero coefficient if added to the model), add the term with the smallest p-value and repeat this step; otherwise, go to step 3.

    3. If any of the available terms in the model have p-values greater than an exit tolerance (that is, the hypothesis of a zero coefficient cannot be rejected), remove the term with the largest p-value and return to step 2; otherwise, end the process.

    At any stage, the function will not add a higher-order term if the model does not also include all lower-order terms that are subsets of the higher-order term. For example, the function will not try to add the term X1:X2^2 unless both X1 and X2^2 are already in the model. Similarly, the function will not remove lower-order terms that are subsets of higher-order terms that remain in the model. For example, the function will not try to remove X1 or X2^2 if X1:X2^2 remains in the model.

    The default value of 'Criterion' for a generalized linear model is 'Deviance'. stepwiseglm and step of GeneralizedLinearModel follow a similar procedure for adding or removing terms.

    You can specify other criteria by using the 'Criterion' name-value pair argument. For example, you can specify the change in the value of the Akaike information criterion, Bayesian information criterion, R-squared, or adjusted R-squared as the criterion to add or remove terms.

    Depending on the terms included in the initial model, and the order in which the function adds and removes terms, the function might build different models from the same set of potential terms. The function terminates when no single step improves the model. However, a different initial model or a different sequence of steps does not guarantee a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.

  • step treats a categorical predictor as follows:

    • A model with a categorical predictor that has L levels (categories) includes L – 1 indicator variables. The model uses the first category as a reference level, so it does not include the indicator variable for the reference level. If the data type of the categorical predictor is categorical, then you can check the order of categories by using categories and reorder the categories by using reordercats to customize the reference level. For more details about creating indicator variables, see Automatic Creation of Dummy Variables.

    • step treats the group of L – 1 indicator variables as a single variable. If you want to treat the indicator variables as distinct predictor variables, create indicator variables manually by using dummyvar. Then use the indicator variables, except the one corresponding to the reference level of the categorical variable, when you fit a model. For the categorical predictor X, if you specify all columns of dummyvar(X) and an intercept term as predictors, then the design matrix becomes rank deficient.

    • Interaction terms between a continuous predictor and a categorical predictor with L levels consist of the element-wise product of the L – 1 indicator variables with the continuous predictor.

    • Interaction terms between two categorical predictors with L and M levels consist of the (L – 1)*(M – 1) indicator variables to include all possible combinations of the two categorical predictor levels.

    • You cannot specify higher-order terms for a categorical predictor because the square of an indicator is equal to itself.

    Therefore, if step adds or removes a categorical predictor, the function actually adds or removes the group of indicator variables in one step. Similarly, if step adds or removes an interaction term with a categorical predictor, the function actually adds or removes the group of interaction terms including the categorical predictor.

  • step considers NaN, '' (empty character vector), "" (empty string), <missing>, and <undefined> values in tbl, X, and Y to be missing values. step does not use observations with missing values in the fit. The ObservationInfo property of a fitted model indicates whether or not step uses each observation in the fit.

Alternative Functionality

  • Use stepwiseglm to specify terms in a starting model and continue improving the model until no single step of adding or removing a term is beneficial.

  • Use addTerms or removeTerms to add or remove specific terms.

Extended Capabilities

Version History

Introduced in R2012a

See Also

| | |

Topics