Main Content

kfoldMargin

Classification margins for cross-validated classification model

collapse all in page

    Syntax

    M = kfoldMargin(CVMdl)
    M = kfoldMargin(CVMdl,'IncludeInteractions',includeInteractions)

    Description

    M = kfoldMargin(CVMdl) returns classification margins obtained by the cross-validated classification model CVMdl. For every fold, kfoldMargin computes classification margins for validation-fold observations using a classifier trained on training-fold observations. CVMdl.X and CVMdl.Y contain both sets of observations.

    example

    M = kfoldMargin(CVMdl,'IncludeInteractions',includeInteractions) specifies whether to include interaction terms in computations. This syntax applies only to generalized additive models.

    Examples

    collapse all

    Open Live Script

    Find the k-fold margins for an ensemble that classifies the ionosphere data.

    Load the ionosphere data set.

    load ionosphere

    Create a template tree stump.

    t = templateTree('MaxNumSplits',1);

    Train a classification ensemble of decision trees. Specify t as the weak learner.

    Mdl = fitcensemble(X,Y,'Method','AdaBoostM1','Learners',t);

    Cross-validate the classifier using 10-fold cross-validation.

    cvens = crossval(Mdl);

    Compute the k-fold margins. Display summary statistics for the margins.

    m = kfoldMargin(cvens);
marginStats = table(min(m),mean(m),max(m),...
    'VariableNames',{'Min','Mean','Max'})
    marginStats=1×3 table
      Min       Mean      Max  
    _______    ______    ______

    -11.312    7.3236    23.517

    Input Arguments

    collapse all

    Cross-validated partitioned classifier, specified as a ClassificationPartitionedModel, ClassificationPartitionedEnsemble, or ClassificationPartitionedGAM object. You can create the object in two ways:

    • Pass a trained classification model listed in the following table to its crossval object function.

    • Train a classification model using a function listed in the following table and specify one of the cross-validation name-value arguments for the function.

    Classification ModelFunction
    ClassificationDiscriminantfitcdiscr
    ClassificationEnsemblefitcensemble
    ClassificationGAMfitcgam
    ClassificationKNNfitcknn
    ClassificationNaiveBayesfitcnb
    ClassificationNeuralNetworkfitcnet
    ClassificationSVMfitcsvm
    ClassificationTreefitctree

    Flag to include interaction terms of the model, specified as true or false. This argument is valid only for a generalized additive model (GAM). That is, you can specify this argument only when CVMdl is ClassificationPartitionedGAM.

    The default value is true if the models in CVMdl (CVMdl.Trained) contain interaction terms. The value must be false if the models do not contain interaction terms.

    Data Types: logical

    Output Arguments

    collapse all

    Classification margins, returned as a numeric vector. M is an n-by-1 vector, where each row is the margin of the corresponding observation and n is the number of observations. (n is size(CVMdl.X,1) when observations are in rows.)

    If you use a holdout validation technique to create CVMdl (that is, if CVMdl.KFold is 1), then M has NaN values for training-fold observations.

    More About

    collapse all

    Classification Margin

    The classification margin for binary classification is, for each observation, the difference between the classification score for the true class and the classification score for the false class. The classification margin for multiclass classification is the difference between the classification score for the true class and the maximal score for the false classes.

    If the margins are on the same scale (that is, the score values are based on the same score transformation), then they serve as a classification confidence measure. Among multiple classifiers, those that yield greater margins are better.

    Algorithms

    kfoldMargin computes classification margins as described in the corresponding margin object function. For a model-specific description, see the appropriate margin function reference page in the following table.

    Model Typemargin Function
    Discriminant analysis classifiermargin
    Ensemble classifiermargin
    Generalized additive model classifiermargin
    k-nearest neighbor classifiermargin
    Naive Bayes classifiermargin
    Neural network classifiermargin
    Support vector machine classifiermargin
    Binary decision tree for multiclass classificationmargin

    Extended Capabilities

    Version History

    Introduced in R2011a

    expand all

    See Also

    | | | |