Doubts about cross-validation

1 次查看(过去 30 天)
I'm trying to test performance of a KNN and a svm (cecoc) classifier for a multiclass problem.
For the knn model I do it by building mdlknn, which is a cross validated model (leave one out) and then using kfoldLoss function
mdlknn = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'Leaveout','on',...
'HyperparameterOptimizationOptions','UseParallel');
perf = 1-kfoldLoss(mdlknn);
Is this correct? My doubts regards the cross-bvalidation:
Using option ('Leaveout','on') in fitcknn is the same as not using it and then calling crossval on the trained model?
mdlknnNoCrossVal = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'HyperparameterOptimizationOptions','UseParallel');
mdlknn = crossval(mdlknnNoCrossVal, 'leaveout','on')
perf = 1-kfoldLoss(mdlknn);
Next, since in knn classifier I'm using my distance, I would like to do the same with the cecoc. How can I do that?
At this stage, the code I use for building the cecoc doesn't use it. how can I set it?
t = templateSVM('Standardize',true,'KernelFunction','rbf');
mdlsvm = fitcecoc(X,labels, 'Leaveout','on',...
'HyperparameterOptimizationOptions','UseParallel',...
'coding', 'ternarycomplete','Learners',t);
perf = 1-kfoldLoss(mdlsvm);
And again: Using option ('Leaveout','on') in fitcecoc is the same as not using it and then calling crossval on the trained model, right?
THANKS A LOT

采纳的回答

Jyothis Gireesh
Jyothis Gireesh 2019-8-22
I am assuming that you want to have some clarification regarding the cross-validation syntax in KNN and SVM
  • KNN Classifier
mdlknn = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'Leaveout','on',...
'HyperparameterOptimizationOptions','UseParallel');
mdlknnNoCrossVal = fitcknn(X,labels, 'NumNeighbors', ...
k, 'Distance',@distKNN, 'HyperparameterOptimizationOptions','UseParallel');
mdlknn = crossval(mdlknnNoCrossVal, 'leaveout','on')
These two syntaxes may be equivalent for the creation of a cross-validated KNN model. “mdlknn" is a "ClassificationPartitionModel" classifier. "mdlknnNoCrossVal” creates a "ClassificationKNN" classifier which is cross validated using the “crossval” function.
  • ECOC classifier
In the case of ECOC classifier you may use the templateKNN as the learner. It provides the feature to customize the distance metric using the ‘Distance’ argument. The distance metric can be passed as a function handle like that of the KNN classifier syntax.
And the two syntaxes are valid for the process of cross validation in the case of ECOC also.

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Classification Ensembles 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by