Confusion matrix of SVM classifier with k-fold cross-validation

Question

Vinícius Ludwig Barbosa 2020-12-2

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/674468-confusion-matrix-of-svm-classifier-with-k-fold-cross-validation

编辑： Vinícius Ludwig Barbosa 2020-12-26

在 MATLAB Online 中打开

I am using fitcsvm to train a SVM model using k-fold cross-validation.

I would like to have access to the observations in predictions which caused FN and FP.

Therefore, I created some code to get the indexes of these observations.

However, I found out that the sum of FN, FP, TN and TP from the confusion matrices related to each

kSVMModel.Trained{k} is not equal to the confusion matrix based on "predictions".

Weren't they supposed to be the same?

c = cvpartition(fullDataY, 'KFold', 10); % create stratified folds
kSVMModel = fitcsvm(fullDataX, fullDataY, 'Standardize', true, 'CVPartition', c);
scorekSVMModel = fitSVMPosterior(kSVMModel);
[predictions, post_scores] = kfoldPredict(scorekSVMModel);
for jj = 1:kSVMModel.KFold % debug
        
        indTrainFold{jj} = find(training(c,jj)==1);
        
        indTestFold{jj} = find(test(c,jj)==1);
        
        [predFold{jj}] = predict(kSVMModel.Trained{jj}, fullDataX(indTestFold{jj},:));
        
        cmFold = confusionchart(fullDataY(indTestFold{jj},:), predFold{jj});
        
        TN(jj) = cmFold.NormalizedValues(1,1);
        
        TP(jj) = cmFold.NormalizedValues(2,2);
        
        FP(jj) = cmFold.NormalizedValues(1,2);
        
        FN(jj) = cmFold.NormalizedValues(2,1);
        
        close all;
         
end
cm = confusionchart(fullDataY, predictions);
sum(TN) == cm.NormalizedValues(1,1);
sum(TP) == cm.NormalizedValues(2,2);
sum(FP) == cm.NormalizedValues(1,2);
sum(FN) == cm.NormalizedValues(2,1);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Aditya Patil 2020-12-22

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/674468-confusion-matrix-of-svm-classifier-with-k-fold-cross-validation#answer_581635

在 MATLAB Online 中打开

You can use confusionmat for getting the confusion matrix. This way, the results are correct. Check the following sample code,

%Generate data
X = rand(100, 1);
Y = [X(:,1) > 0.5];
% Fit svm model
cvp = cvpartition(Y, 'KFold', 4);
mdl = fitcsvm(X,Y, 'CVPartition', cvp);
prediction = kfoldPredict(mdl);
confusionmat(prediction, Y)
% compare with individual results
FoldPredictions = zeros(mdl.KFold, 2, 2);
for counter = 1: mdl.KFold
    index = test(cvp, counter);
    predictFolds = predict(mdl.Trained{counter}, X(index));
    FoldPredictions(counter,:,:) = confusionmat(predictFolds, Y(index));
end
sum(FoldPredictions, 1)