ROC curve for the validation set

Question

Uerm 2020-1-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/500122-roc-curve-for-the-validation-set

评论： Silvia 2020-2-19

Hi,

I am working on a classification problem, where I use 10-fold cross-validation. I have made the code for the ROC curve for the training part of the data. I want to do the same for the validation set. How can I do that? My code is below:

indices = crossvalind('Kfold',trainingData(:,end),10);
for i = 1:10
    test = (indices == i); 
    train = ~test;
    
    % Linear SVM
    classificationLinearSVM128 = fitcsvm(...
    trainingData(train,1:end-1),...
    trainingData(train,end), ...
    'KernelFunction', 'linear', ...
    'PolynomialOrder', [], ...
    'KernelScale', 'auto', ...
    'BoxConstraint', 1, ...
    'Standardize', true, ...
    'ClassNames', [0; 1]);
    % Training
    [predsLinSVM128train,~] = predict(classificationLinearSVM128,trainingData(train,1:end-1));
    targetsLinSVM128train = trainingData(train,end);
    [~,scoresLinSVM128] = resubPredict(fitPosterior(classificationLinearSVM128));
    [xLinSVM128,yLinSVM128,~,aucLinSVM128] = perfcurve(trainingData(train,end),scoresLinSVM128(:,2),1);
    
    % Validation
    [predsLinSVM128test,~] = predict(classificationLinearSVM128,trainingData(test,1:end-1));
    targetsLinSVM128test = trainingData(test,end);
   
    [~,scoresLinSVM128test] = resubPredict(fitPosterior(classificationLinearSVM128));
    %[xLinSVM128test,yLinSVM128test,~,aucLinSVM128test] = perfcurve(trainingData(test,end),scoresLinSVM128test(:,2),1);
end
figure()
subplot(121)
confusionchart(targetsLinSVM128train,predsLinSVM128train)
title('Linear SVM, training')
subplot(122)
confusionchart(targetsLinSVM128test,predsLinSVM128test)
title('Linear SVM, validation')
figure()
plot(xLinSVM128,yLinSVM128,'LineWidth',2)
xlabel('False Positive Rate')
ylabel('True Positive Rate')
title('ROC, Linear SVM')

I have tried the line that I have commented, but it does not work. Can anyone help with this? The positive class is 1.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shashank Gupta 2020-1-21

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/500122-roc-curve-for-the-validation-set#answer_411166

Hi Uerm,

I don’t see any point, why “perfcurve" function should not work. You even used the same function in your training part and it seems to be working there. I am not really sure why it is not working. Also you can verify it by calculating FPR(False positive rate) and TPR(True positive rate) manually or through ROC function, plot the FPR vs TPR and see if it matches with the perfcurve function output.

Also let me know if you find something interesting. It is supposed to work perfectly. If you are getting any error while using the function post that as well.

2 个评论
显示无隐藏无

Uerm 2020-1-24

Hi Shashank,

Thanks for your reply. Fortunately, I did manage to make it work for both the training part and the validation part.

Silvia 2020-2-19

Hi Uerm,

How did you solve the problem?

Thank you

请先登录，再进行评论。

ROC curve for the validation set

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

2 个评论
显示无隐藏无

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

ROC curve for the validation set

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

2 个评论 显示 无隐藏 无

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

2 个评论
显示无隐藏无