Main Content

RegressionSVM Class

Superclasses: CompactRegressionSVM

Support vector machine regression model

expand all in page

Description

RegressionSVM is a support vector machine (SVM) regression model. Train a RegressionSVM model using fitrsvm and the sample data.

RegressionSVM models store data, parameter values, support vectors, and algorithmic implementation information. You can use these models to:

  • Estimate resubstitution predictions. For details, see resubPredict.

  • Predict values for new data. For details, see predict.

  • Compute resubstitution loss. For details, see resubLoss.

  • Compute the mean square error or epsilon-insensitive loss. For details, see loss.

Construction

Create a RegressionSVM object by using fitrsvm.

Properties

expand all

Dual problem coefficients, specified as a vector of numeric values. Alpha contains m elements, where m is the number of support vectors in the trained SVM regression model. The dual problem introduces two Lagrange multipliers for each support vector. The values of Alpha are the differences between the two estimated Lagrange multipliers for the support vectors. For more details, see Understanding Support Vector Machine Regression.

If you specified to remove duplicates using RemoveDuplicates, then, for a particular set of duplicate observations that are support vectors, Alpha contains one coefficient corresponding to the entire set. That is, MATLAB® attributes a nonzero coefficient to one observation from the set of duplicates and a coefficient of 0 to all other duplicate observations in the set.

Data Types: single | double

Primal linear problem coefficients, stored as a numeric vector of length p, where p is the number of predictors in the SVM regression model.

The values in Beta are the linear coefficients for the primal optimization problem.

If the model is obtained using a kernel function other than 'linear', this property is empty ('[]').

The predict method computes predicted response values for the model as YFIT = (X/S)×Beta + Bias, where S is the value of the kernel scale stored in the KernelParameters.Scale property.

Data Types: single | double

Bias term in the SVM regression model, stored as a scalar value.

Data Types: single | double

Box constraints for dual problem alpha coefficients, stored as a numeric vector containing n elements, where n is the number of observations in X (Mdl.NumObservations).

The absolute value of the dual coefficient Alpha for observation i cannot exceed BoxConstraints(i).

If you specify removing duplicates using 'RemoveDuplicates', then for a given set of duplicate observations, MATLAB sums the box constraints, and then attributes the sum to one observation and box constraints of 0 to all other observations in the set.

Data Types: single | double

Caching information, stored as a structure with the following fields.

FieldDescription
'Size'Positive scalar value indicating the cache size (in MB) that the software reserves to store entries of the Gram matrix. Set the cache size by using the 'CacheSize' name-value pair argument in fitrsvm.
'Algorithm'Character vector containing the name of the algorithm used to remove entries from the cache when its capacity is exceeded. Currently, the only available caching algorithm is 'Queue'. You cannot set the caching algorithm.

Data Types: struct

This property is read-only.

Categorical predictor indices, specified as a vector of positive integers. CategoricalPredictors contains index values indicating that the corresponding predictors are categorical. The index values are between 1 and p, where p is the number of predictors used to train the model. If none of the predictors are categorical, then this property is empty ([]).

Data Types: single | double

Convergence information, stored as a structure with the following fields.

FieldDescription
ConvergedLogical flag indicating whether the algorithm converged. A value of 1 indicates convergence.
ReasonForConvergenceCharacter vector indicating the criterion the software used to detect convergence.
GapScalar feasibility gap between the dual and primal objective functions.
GapToleranceScalar tolerance for the feasibility gap. You can set this tolerance using the 'GapTolerance' name-value pair argument in fitrsvm.
DeltaGradientScalar gradient difference between upper and lower violators.
DeltaGradientToleranceScalar tolerance for the gradient difference. You can set this tolerance using the DeltaGradientTolerance name-value pair argument in fitrsvm.
LargestKKTViolationMaximal scalar Karush-Kuhn-Tucker (KKT) violation value.
KKTToleranceScalar tolerance for the largest KKT violation. You can set this tolerance using the 'KKTTolerance' name-value pair argument in fitrsvm.
History

Structure containing convergence information recorded at periodic intervals during the model training process. This structure contains the following fields:

  • NumIterations — Array of iteration indices at which the software recorded convergence criteria.

  • Gap — Gap values at these iterations.

  • DeltaGradient — DeltaGradient values at these iterations.

  • LargestKKTViolation — LargestKKTViolation values at these iterations.

  • NumSupportVectors — Number of support vectors at these iterations.

  • Objective — Objective values at these iterations.

ObjectiveNumeric value of the dual objective.

Data Types: struct

Half the width of the epsilon-insensitive band, stored as a nonnegative scalar value.

Data Types: single | double

Expanded predictor names, stored as a cell array of character vectors.

If the model uses encoding for categorical variables, then ExpandedPredictorNames includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames is the same as PredictorNames.

Data Types: cell

Gradient values in training data, stored as a numeric vector containing 2n elements, where n is the number of observations in the training data.

Element i of Gradient contains the gradient value for the Alpha coefficient that corresponds to the upper boundary of the epsilon-insensitive band at observation i at the end of the optimization.

Element i + NumObservations of Gradient contains the gradient value for the Alpha coefficient that corresponds to the lower boundary of the epsilon-insensitive band at observation i at the end of the optimization.

Data Types: single | double

This property is read-only.

Cross-validation optimization of hyperparameters, specified as a BayesianOptimization object or a table of hyperparameters and associated values. This property is nonempty if the 'OptimizeHyperparameters' name-value pair argument is nonempty when you create the model. The value of HyperparameterOptimizationResults depends on the setting of the Optimizer field in the HyperparameterOptimizationOptions structure when you create the model.

Value of Optimizer OptionValue of HyperparameterOptimizationResults
"bayesopt" (default)Object of class BayesianOptimization
"gridsearch" or "randomsearch"Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst)

Flag indicating whether an observation is a support vector, stored as an n-by-1 logical vector. n is the number of observations in X (see NumObservations). A value of 1 indicates that the corresponding observation in the training data is a support vector.

If you specify removing duplicates using RemoveDuplicates, then for a given set of duplicate observations that are support vectors, IsSupportVector flags only one as a support vector.

Data Types: logical

Kernel function parameters, stored as a structure with the following fields.

FieldDescription
Function Kernel function name (a character vector).
ScaleNumeric scale factor used to divide predictor values.

You can specify values for KernelParameters.Function and KernelParameters.Scale by using the KernelFunction and KernelScale name-value pair arguments in fitrsvm, respectively.

Data Types: struct

Parameter values used to train the SVM regression model, stored as a model parameter object. Access the properties of ModelParameters using dot notation. For example, access the value of Epsilon used to train the model as Mdl.ModelParameters.Epsilon.

Predictor means, stored as a vector of numeric values.

If the predictors are standardized, then Mu is a numeric vector of length p, where p is the number of predictors used to train the model. In this case, the predict method centers predictor matrix X by subtracting the corresponding element of Mu from each column.

If the predictors are not standardized, then Mu is empty ('[]').

If the data contains categorical predictors, then Mu includes elements for the dummy variables for those predictors. The corresponding entries in Mu are 0 because dummy variables are not centered or scaled.

Data Types: single | double

Number of iterations required for the optimization routine to reach convergence, stored as a positive integer value.

To set a limit on the number of iterations, use the 'IterationLimit' name-value pair argument of fitrsvm.

Data Types: single | double

Number of observations in the training data, stored as a positive integer value.

Data Types: single | double

Predictor names, stored as a cell array of character vectors containing the name of each predictor in the order they appear in X. PredictorNames has a length equal to the number of columns in X.

Data Types: cell

Expected fraction of outliers in the training set, stored as a scalar value in the range [0,1]. You can specify the expected fraction of outliers using the 'OutlierFraction' name-value pair argument in fitrsvm.

Data Types: double

Response variable name, stored as a character vector.

Data Types: char

Response transformation function, specified as 'none' or a function handle. ResponseTransform describes how the software transforms raw response values.

For a MATLAB function or a function that you define, enter its function handle. For example, you can enter Mdl.ResponseTransform = @function, where function accepts a numeric vector of the original responses and returns a numeric vector of the same size containing the transformed responses.

Data Types: char | function_handle

Rows of the original training data stored in the model, specified as a logical vector. This property is empty if all rows are stored in X and Y.

Data Types: logical

Number of iterations between reductions of the active set during optimization, stored as a nonnegative integer value.

You can set the shrinkage period by using the 'ShrinkagePeriod' name-value pair argument in fitrsvm.

Data Types: single | double

Predictor standard deviations, stored as a vector of numeric values.

If the predictors are standardized, then Sigma is a numeric vector of length p, where p is the number of predictors used to train the model. In this case, the predict method scales the predictor matrix X by dividing each column by the corresponding element of Sigma, after centering each element using Mu.

If the predictors are not standardized, then Sigma is empty ('[]').

If the data contains categorical predictors, Sigma includes elements for the dummy variables for those predictors. The corresponding entries in Sigma are 1, because dummy variables are not centered or scaled.

Data Types: single | double

Name of the solver algorithm used to solve the optimization problem, stored as a value in this table.

ValueDescription
'SMO'Sequential Minimal Optimization
'ISDA'Iterative Single Data Algorithm
'L1QP'L1 soft-margin minimization by quadratic programming (requires an Optimization Toolbox™ license).

Support vectors, stored as an m-by-p matrix of numeric values. m is the number of support vectors (sum(Mdl.IsSupportVector)), and p is the number of predictors in X.

If you specified to remove duplicates using RemoveDuplicates, then for a given set of duplicate observations that are support vectors, SupportVectors contains one unique support vector.

Data Types: single | double

Observation weights used to train the model, stored as a numeric vector containing NumObservation number of elements. fitrsvm normalizes the weights used for training so that they sum to 1.

Data Types: single | double

Predictor values used to train the model, stored as a matrix of numeric values if the model is trained on a matrix, or a table if the model is trained on a table. X has size n-by-p, where n is the number of rows and p is the number of predictor variables or columns in the training data.

Data Types: single | double | table

Observed response values, stored as a numeric vector containing NumObservations number of elements.

Data Types: single | double

Object Functions

compactCompact support vector machine regression model
crossvalCross-validate machine learning model
discardSupportVectorsDiscard support vectors for linear support vector machine (SVM) regression model
gatherGather properties of Statistics and Machine Learning Toolbox object from GPU
incrementalLearnerConvert support vector machine (SVM) regression model to incremental learner
limeLocal interpretable model-agnostic explanations (LIME)
lossRegression error for support vector machine regression model
partialDependenceCompute partial dependence
plotPartialDependenceCreate partial dependence plot (PDP) and individual conditional expectation (ICE) plots
predictPredict responses using support vector machine regression model
resubLossResubstitution loss for support vector machine regression model
resubPredictPredict resubstitution response of support vector machine regression model
resumeResume training support vector machine regression model
shapleyShapley values

Copy Semantics

Value. To learn how value classes affect copy operations, see Copying Objects.

Examples

collapse all

Open Live Script

This example shows how to train a linear support vector machine (SVM) regression model using sample data stored in matrices.

Load the carsmall data set.

load carsmall
rng default  % for reproducibility

Specify Horsepower and Weight as the predictor variables (X) and MPG as the response variable (Y).

X = [Horsepower,Weight];
Y = MPG;

Train a linear SVM regression model.

Mdl = fitrsvm(X,Y)
Mdl = 
  RegressionSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
        ResponseTransform: 'none'
                    Alpha: [75x1 double]
                     Bias: 57.3800
         KernelParameters: [1x1 struct]
          NumObservations: 94
           BoxConstraints: [94x1 double]
          ConvergenceInfo: [1x1 struct]
          IsSupportVector: [94x1 logical]
                   Solver: 'SMO'

The Command Window shows that Mdl is a trained RegressionSVM model and a list of its properties.

Check the model for convergence.

Mdl.ConvergenceInfo.Converged
ans = logical
   0

0 indicates that the model did not converge.

MdlStd = fitrsvm(X,Y,'Standardize',true)
MdlStd = 
  RegressionSVM
             ResponseName: 'Y'
    CategoricalPredictors: []
        ResponseTransform: 'none'
                    Alpha: [77x1 double]
                     Bias: 22.9131
         KernelParameters: [1x1 struct]
                       Mu: [109.3441 2.9625e+03]
                    Sigma: [45.3545 805.9668]
          NumObservations: 94
           BoxConstraints: [94x1 double]
          ConvergenceInfo: [1x1 struct]
          IsSupportVector: [94x1 logical]
                   Solver: 'SMO'

Check the model for convergence.

MdlStd.ConvergenceInfo.Converged
ans = logical
   1

1 indicates that the model did converge.

Compute the resubstitution mean squared error for the new model.

lStd = resubLoss(MdlStd)
lStd = 
16.8551
Open Live Script

Train a support vector machine regression model using the abalone data from the UCI Machine Learning Repository.

Download the data and save it in your current folder with the name 'abalone.csv'.

url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data';
websave('abalone.csv',url);

Read the data into a table. Specify the variable names.

varnames = {'Sex'; 'Length'; 'Diameter'; 'Height'; 'Whole_weight';...
    'Shucked_weight'; 'Viscera_weight'; 'Shell_weight'; 'Rings'};
Tbl = readtable('abalone.csv','Filetype','text','ReadVariableNames',false);
Tbl.Properties.VariableNames = varnames;

The sample data contains 4177 observations. All the predictor variables are continuous except for Sex, which is a categorical variable with possible values 'M' (for males), 'F' (for females), and 'I' (for infants). The goal is to predict the number of rings (stored in Rings) on the abalone and determine its age using physical measurements.

Train an SVM regression model, using a Gaussian kernel function with an automatic kernel scale. Standardize the data.

rng default  % For reproducibility
Mdl = fitrsvm(Tbl,'Rings','KernelFunction','gaussian','KernelScale','auto',...
    'Standardize',true)
Mdl = 
  RegressionSVM
           PredictorNames: {'Sex'  'Length'  'Diameter'  'Height'  'Whole_weight'  'Shucked_weight'  'Viscera_weight'  'Shell_weight'}
             ResponseName: 'Rings'
    CategoricalPredictors: 1
        ResponseTransform: 'none'
                    Alpha: [3635×1 double]
                     Bias: 10.8144
         KernelParameters: [1×1 struct]
                       Mu: [0 0 0 0.5240 0.4079 0.1395 0.8287 0.3594 0.1806 0.2388]
                    Sigma: [1 1 1 0.1201 0.0992 0.0418 0.4904 0.2220 0.1096 0.1392]
          NumObservations: 4177
           BoxConstraints: [4177×1 double]
          ConvergenceInfo: [1×1 struct]
          IsSupportVector: [4177×1 logical]
                   Solver: 'SMO'


  Properties, Methods

The Command Window shows that Mdl is a trained RegressionSVM model and displays a property list.

Display the properties of Mdl using dot notation. For example, check to confirm whether the model converged and how many iterations it completed.

conv = Mdl.ConvergenceInfo.Converged
conv = logical
   1


iter = Mdl.NumIterations
iter = 2759

The returned results indicate that the model converged after 2759 iterations.

References

[1] Nash, W.J., T. L. Sellers, S. R. Talbot, A. J. Cawthorn, and W. B. Ford. "The Population Biology of Abalone (Haliotis species) in Tasmania. I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait." Sea Fisheries Division, Technical Report No. 48, 1994.

[2] Waugh, S. "Extending and Benchmarking Cascade-Correlation: Extensions to the Cascade-Correlation Architecture and Benchmarking of Feed-forward Supervised Artificial Neural Networks." University of Tasmania Department of Computer Science thesis, 1995.

[3] Clark, D., Z. Schreter, A. Adams. "A Quantitative Comparison of Dystal and Backpropagation." submitted to the Australian Conference on Neural Networks, 1996.

[4] Lichman, M. UCI Machine Learning Repository, [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

Extended Capabilities

Version History

Introduced in R2015b

expand all

See Also

| |

Topics