How to do a ROC analysis using Matlab build-in SVM (Not LibSVM)

Question

Aaronne 2013-3-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/68138-how-to-do-a-roc-analysis-using-matlab-build-in-svm-not-libsvm

回答： Mehrdad Heydarzadeh 2016-8-10

Hi all,

Just wondering anyone knows how to do a ROC analysis using Matlab build-in SVM? This question has been asked by millions of times on the web, but no answer.

svmStruct                               = svmtrain(featureSelcted, groundTruthGroup, 'Kernel_Function', 'rbf', 'Method', 'QP');
svmClassified                           = svmclassify(svmStruct,featureSelcted);
% Predict resubstitution response of SVM classifier
SVMScore ???
% Fit probabilities for scores
[FPR, TPR, Thr, AUC, OPTROCPT]          = perfcurve(groundTruth(:,1), SVMScore(:,1), 1);

Essentially, we need a function to get the '*scores*' of the SVM classifier (SVMScore). Thanks!

A.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Ilya 2013-3-21

7
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/68138-how-to-do-a-roc-analysis-using-matlab-build-in-svm-not-libsvm#answer_79494

在 MATLAB Online 中打开

Well, there is an answer here http://www.mathworks.com/matlabcentral/answers/64475-does-anybody-have-expertise-with-matlab-svm-classifier, with a reference to another thread. I am collecting all the pieces in one place.

Assume your class labels are -1 and +1, assume that you have trained with 'autoscale' set to true by default, let svm be the struct for the trained SVM model, and let Xnew be the new data for which you need to compute the soft scores.

shift = svm.ScaleData.shift;
scale = svm.ScaleData.scaleFactor;
Xnew = bsxfun(@plus,Xnew,shift);
Xnew = bsxfun(@times,Xnew,scale);
sv = svm.SupportVectors;
alphaHat = svm.Alpha;
bias = svm.Bias;
kfun = svm.KernelFunction;
kfunargs = svm.KernelFunctionArgs;
f = kfun(sv,Xnew,kfunargs{:})'*alphaHat(:) + bias;
f = -f; % flip the sign to get the score for the +1 class

Then call perfcurve(true_labels,f,1).

7 个评论
显示 5更早的评论隐藏 5更早的评论

Aaronne 2013-3-22

在 MATLAB Online 中打开

Hi Ilya,

That's quite handy. Thanks so much. But more detailed questions:

1. I have noticed that the predict scores are posterior probabilities for classes obj predicts for the discriminant classification methods in Matlab like KNN and LDA, for example:

%%Construct a LDA classifier
LDAClassifierObject = ClassificationDiscriminant.fit(featureSelcted, groundTruthGroup, 'DiscrimType', 'linear');
% Predict resubstitution response of LDA classifier
[LDALabel, LDAScore] = resubPredict(LDAClassifierObject);
% Fit probabilities for scores
[FPR, TPR, Thr, AUC, OPTROCPT] = perfcurve(groundTruthNumericalLable(:,1), LDAScore(:,1), 1);

Then the scores, ie, LDAScore stores all the posterior probabilities ranged within [0,1].

However, the scores calculated by the svmStruct (the struct for the trained SVM model) has many values larger than 1 and some negitive values. Should we do some normalization for this? For example:

SVMScore = kfun(sv,Xnew,kfunargs{:})'*alphaHat(:) + bias;
SVMScore = SVMScore./max(SVMScore);

2. I am not sure why we need this flipping step:

SVMScore = kfun(sv,Xnew,kfunargs{:})'*alphaHat(:) + bias;
SVMScore = -SVMScore; % flip the sign to get the score for the +1 class

It seems to me that I define the ground truth of the first class to label 1 and the second class to label -1. I don't need the flipping.

3. Also want to make sure that the 'Xnew' variable you used is the 'training data'. For the cross-validation case, 'Xnew' is the training data in each fold, am I right?

4. If you have time, could you please explain a little bit more about this equation:

SVMScore = kfun(sv,Xnew,kfunargs{:})'*alphaHat(:) + bias;

Where did Matlab documents mentioned about this? Otherwise, I will google to see if I can fully understand this.

Looking forward to your replies. Thanks very much again.

A.

Tahir 2015-5-8

在 MATLAB Online 中打开

Hi, i just use the code ...but i have confusion about the line

f = -f; % flip the sign to get the score for the +1 class
why this flipping?? i use labels 0 and 1...still i should flip the f.??

Farag Zbeda 2015-6-3

Hi Ilya

What do yo mean by true_labels when calling perfcurve(true_labels,f,1)? I guess they are the test instances labels. If so, how to calculate them?

Thanks

Faraj

请先登录，再进行评论。

Answer 2

Mehrdad Heydarzadeh 2016-8-10

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/68138-how-to-do-a-roc-analysis-using-matlab-build-in-svm-not-libsvm#answer_231387

Hi Aaronne, It seems that your problem has been answered. But there is a shorter way of doing that using Matlab functions. Just have a look at the second example in following link: http://www.mathworks.com/help/stats/perfcurve.html