SVM cross validation folds' accuracy

Question

Uyen Pham 2020-2-6

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/504120-svm-cross-validation-folds-accuracy

回答： chen study 2024-1-17

I am trying to extract each cross validation fold's accuracy from SVM Gauss med model provided on MatLab's App. 
For example, when I choose 5 fold of cross validation, there are should be 5 accuracy number returned. I use SVM med Gauss. 
Here is the code I export from model, I thought i would go inside this function to manual
code for the cross validation data but cant figure out how. Please help! Thank you
function [trainedClassifier, validationAccuracy] = SVMmediumGauss(trainingData)
% [trainedClassifier, validationAccuracy] = trainClassifier(trainingData)
% returns a trained classifier and its accuracy. This code recreates the
% classification model trained in Classification Learner app. Use the
% generated code to automate training the same model with new data, or to
% learn how to programmatically train models.
%
%  Input:
%      trainingData: a table containing the same predictor and response
%       columns as imported into the app.
%
%  Output:
%      trainedClassifier: a struct containing the trained classifier. The
%       struct contains various fields with information about the trained
%       classifier.
%
%      trainedClassifier.predictFcn: a function to make predictions on new
%       data.
%
%      validationAccuracy: a double containing the accuracy in percent. In
%       the app, the History list displays this overall accuracy score for
%       each model.
%
% Use the code to train the model with new data. To retrain your
% classifier, call the function from the command line with your original
% data or new data as the input argument trainingData.
%
% For example, to retrain a classifier trained with the original data set
% T, enter:
%   [trainedClassifier, validationAccuracy] = trainClassifier(T)
%
% To make predictions with the returned 'trainedClassifier' on new data T2,
% use
%   yfit = trainedClassifier.predictFcn(T2)
%
% T2 must be a table containing at least the same predictor columns as used
% during training. For details, enter:
%   trainedClassifier.HowToPredict
% Auto-generated by MATLAB on 05-Feb-2020 12:30:08
% Extract predictors and response
% This code processes the data into the right shape for training the
% model.
inputTable = trainingData;
predictorNames = {'FSCA', 'FSCH', 'FSCW', 'SSCA', 'SSCH', 'SSCW'};
predictors = inputTable(:, predictorNames);
response = inputTable.STATE;
isCategoricalPredictor = [false, false, false, false, false, false];
% Train a classifier
% This code specifies all the classifier options and trains the classifier.
classificationSVM = fitcsvm(...
    predictors, ...
    response, ...
    'KernelFunction', 'gaussian', ...
    'PolynomialOrder', [], ...
    'KernelScale', 2.4, ...
    'BoxConstraint', 1, ...
    'Standardize', true, ...
    'ClassNames', categorical({'APOP'; 'LIVE'}));
% Create the result struct with predict function
predictorExtractionFcn = @(t) t(:, predictorNames);
svmPredictFcn = @(x) predict(classificationSVM, x);
trainedClassifier.predictFcn = @(x) svmPredictFcn(predictorExtractionFcn(x));
% Add additional fields to the result struct
trainedClassifier.RequiredVariables = {'FSCA', 'FSCH', 'FSCW', 'SSCA', 'SSCH', 'SSCW'};
trainedClassifier.ClassificationSVM = classificationSVM;
trainedClassifier.About = 'This struct is a trained model exported from Classification Learner R2019b.';
trainedClassifier.HowToPredict = sprintf('To make predictions on a new table, T, use: \n  yfit = c.predictFcn(T) \nreplacing ''c'' with the name of the variable that is this struct, e.g. ''trainedModel''. \n \nThe table, T, must contain the variables returned by: \n  c.RequiredVariables \nVariable formats (e.g. matrix/vector, datatype) must match the original training data. \nAdditional variables are ignored. \n \nFor more information, see <a href="matlab:helpview(fullfile(docroot, ''stats'', ''stats.map''), ''appclassification_exportmodeltoworkspace'')">How to predict using an exported model</a>.');
% Extract predictors and response
% This code processes the data into the right shape for training the
% model.
inputTable = trainingData;
predictorNames = {'FSCA', 'FSCH', 'FSCW', 'SSCA', 'SSCH', 'SSCW'};
predictors = inputTable(:, predictorNames);
response = inputTable.STATE;
isCategoricalPredictor = [false, false, false, false, false, false];
% Perform cross-validation
partitionedModel = crossval(trainedClassifier.ClassificationSVM, 'KFold', 5);
% Compute validation predictions
[validationPredictions, validationScores] = kfoldPredict(partitionedModel);
% Compute validation accuracy
validationAccuracy = 1 - kfoldLoss(partitionedModel, 'LossFun', 'ClassifError');
end
%[trainClassifier, validationAccury]=SVMmediumGauss(trainingsinglet1)

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Bhargavi Maganuru 2020-2-13

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/504120-svm-cross-validation-folds-accuracy#answer_415348

If you use k-fold cross-validation, for each fold, app trains the model using out-of-fold observations and tests on the in-fold data and discards the model. Final score is the average accuracy score over all folds and the final model is always trained using full data set.

For more information you can refer following links

https://www.mathworks.com/help/stats/select-data-and-validation-for-classification-problem.html#bunsu7g-1

https://www.mathworks.com/help/stats/assess-classifier-performance.html