KNN classifier with ROC Analysis

Question

Aaronne 2013-3-19

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/67790-knn-classifier-with-roc-analysis

Hi Smart guys,

I wrote following codes to get a plot of ROC for my KNN classifier:

    load fisheriris;
    features                                = meas;
    featureSelcted                          = features;
    numFeatures                             = size(meas,1);
    %%Define ground truth
    groundTruthGroup                        = species;
    %%Construct a KNN classifier
    KNNClassifierObject                     = ClassificationKNN.fit(featureSelcted, groundTruthGroup, 'NumNeighbors', 3, 'Distance', 'euclidean');
    % Predict resubstitution response of k-nearest neighbor classifier
    [KNNLabel, KNNScore]                     = resubPredict(KNNClassifierObject);
    % Fit probabilities for scores
    groundTruthNumericalLable          = [ones(50,1); zeros(50,1); -1.*ones(50,1)];
    [FPR, TPR, Thr, AUC, OPTROCPT]          = perfcurve(groundTruthNumericalLable(:,1), KNNScore(:,1), 1);

Then we can plot the FPR vs TPR to get the ROC curve.

However, the FPR and TPR is different from what I got using my own implementation that the one above will not display all the points, actually, the codes above display only three points on the ROC. The codes I implemented will dispaly 151 points on the ROC as the size of the data is 150.

    patternsKNN                             = [KNNScore(:,1), groundTruthNumericalLable(:,1)];
    patternsKNN                             = sortrows(patternsKNN, -1);
    groundTruthPattern                      = patternsKNN(:,2);
    POS                                     = cumsum(groundTruthPattern==1);
    TPR                                     = POS/sum(groundTruthPattern==1);
    NEG                                     = cumsum(groundTruthPattern==0);
    FPR                                     = NEG/sum(groundTruthPattern==0);
    FPR                                     = [0; FPR];
    TPR                                     = [0; TPR];

May I ask how to tune '`perfcurve`' to let it output all the points for the ROC? Thanks a lot.

A.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Alessandro 2013-3-20

编辑：Alessandro 2013-3-20

try adding 'xvals','all' [FPR, TPR, Thr, AUC, OPTROCPT] = perfcurve(groundTruthNumericalLable(:,1), KNNScore(:,1), 1,'xvals','all');

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ilya 2013-3-19

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/67790-knn-classifier-with-roc-analysis#answer_79273

For 3 neighbors, the posterior probability has at most 4 distinct values, namely (0:3)/3. Likely less for the Fisher iris data because the classes are well separated. With 4 distinct score values, you won't see more than 4 points on the ROC curve. Your implementation does not account for such ties.

2 个评论
显示无隐藏无

Aaronne 2013-3-20

Hi Ilya,

Thanks for your reply. Does that mean my implementation is wrong?

Why we we can't have more than 4 points on the ROC curve if there are 4 distinct score values? I thought the number of points on the ROC curve is defined as size of the data plus one.

A.

Ilya 2013-3-20

Yes, it does mean that your implementation is wrong. As I said, you can't have more points on a ROC curve than distinct threshold values. This is actually quite simple - you just need to think about it.

请先登录，再进行评论。