ROC curve for the validation set

Question

Uerm 2020-1-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/500122-roc-curve-for-the-validation-set

评论： Silvia 2020-2-19

Hi,

I am working on a classification problem, where I use 10-fold cross-validation. I have made the code for the ROC curve for the training part of the data. I want to do the same for the validation set. How can I do that? My code is below:

indices = crossvalind('Kfold',trainingData(:,end),10);
for i = 1:10
    test = (indices == i); 
    train = ~test;
    
    % Linear SVM
    classificationLinearSVM128 = fitcsvm(...
    trainingData(train,1:end-1),...
    trainingData(train,end), ...
    'KernelFunction', 'linear', ...
    'PolynomialOrder', [], ...
    'KernelScale', 'auto', ...
    'BoxConstraint', 1, ...
    'Standardize', true, ...
    'ClassNames', [0; 1]);
    % Training
    [predsLinSVM128train,~] = predict(classificationLinearSVM128,trainingData(train,1:end-1));
    targetsLinSVM128train = trainingData(train,end);
    [~,scoresLinSVM128] = resubPredict(fitPosterior(classificationLinearSVM128));
    [xLinSVM128,yLinSVM128,~,aucLinSVM128] = perfcurve(trainingData(train,end),scoresLinSVM128(:,2),1);
    
    % Validation
    [predsLinSVM128test,~] = predict(classificationLinearSVM128,trainingData(test,1:end-1));
    targetsLinSVM128test = trainingData(test,end);
   
    [~,scoresLinSVM128test] = resubPredict(fitPosterior(classificationLinearSVM128));
    %[xLinSVM128test,yLinSVM128test,~,aucLinSVM128test] = perfcurve(trainingData(test,end),scoresLinSVM128test(:,2),1);
end
figure()
subplot(121)
confusionchart(targetsLinSVM128train,predsLinSVM128train)
title('Linear SVM, training')
subplot(122)
confusionchart(targetsLinSVM128test,predsLinSVM128test)
title('Linear SVM, validation')
figure()
plot(xLinSVM128,yLinSVM128,'LineWidth',2)
xlabel('False Positive Rate')
ylabel('True Positive Rate')
title('ROC, Linear SVM')

I have tried the line that I have commented, but it does not work. Can anyone help with this? The positive class is 1.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shashank Gupta 2020-1-21

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/500122-roc-curve-for-the-validation-set#answer_411166

Hi Uerm,

I don’t see any point, why “perfcurve" function should not work. You even used the same function in your training part and it seems to be working there. I am not really sure why it is not working. Also you can verify it by calculating FPR(False positive rate) and TPR(True positive rate) manually or through ROC function, plot the FPR vs TPR and see if it matches with the perfcurve function output.

Also let me know if you find something interesting. It is supposed to work perfectly. If you are getting any error while using the function post that as well.