Help using perfcurve function!!

2012 8 14
2 个回答
回答已采纳
6 次查看（30 天）

0 个投票

Hey everyone,
I have three groups of people normal ones, one with disease in its early stages and last one people with disease in its severe stages. And in each group I study 16 different features. In order to check the discriminatory performance of several features, I need to use area under the receiver operating characteristic curve. I don't know how can i translate my data to input arguments for the perfcurve function!! [X,Y,T,AUC,OPTROCPT,SUBY,SUBYNAMES] = perfcurve(labels,scores,posclass);
Any help will be appreciated.

1 个评论
显示 -1更早的评论 隐藏 -1更早的评论

Yasmin Tamimi
Yasmin Tamimi 2012-8-14
编辑：Yasmin Tamimi 2012-8-14
here is my code but even after writing it and getting the results I still don't understand what does the numbers mean or interpret for X, Y, and AUC!! and on what bases did we choose to assign the ones and zeros for the labels and if I flip will I get different results!! plz advice
labels = {'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'can-';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'ecan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+';'dcan+'};
data= [Data_normal; BB; CC ]; % the first group for normal people then people with early disease and finally with severe disease
x1= data((1:132),:); %combinations of can- and ecan+
x2= data([1:70 133:144],:); %combinations of can- and dcan+
x3= data((71:end),:); %combinations of ecan+ and dcan
y1 = (1:132)'>70; %% can- =0, ecan+ =1
y2 = (1:82)'>70; %% can- =0, dcan+ =1
y3 = (1:74)'>62; %% ecan+ =0, dcan+ =1
b1 = glmfit(x1,y1,'binomial');
b2 = glmfit(x2,y2,'binomial');
b3 = glmfit(x3,y3,'binomial');
p1 = glmval(b1,x1,'logit');
p2 = glmval(b2,x2,'logit');
p3 = glmval(b3,x3,'logit');
[X1,Y1,T1,AUC1] = perfcurve(labels((1:132),:),p1,'ecan+');
[X2,Y2,T2,AUC2] = perfcurve(labels([1:70 133:144],:),p2,'dcan+');
[X3,Y3,T3,AUC3] = perfcurve(labels((71:end),:),p3,'dcan+');
figure(1)
plot(X1,Y1)
xlabel('False positive rate'); ylabel('True positive rate')
title('ROC for classification by logistic regression')
figure(2)
plot(X2,Y2)
xlabel('False positive rate'); ylabel('True positive rate')
title('ROC for classification by logistic regression')
figure(3)
plot(X3,Y3)
xlabel('False positive rate'); ylabel('True positive rate')
title('ROC for classification by logistic regression')

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

Ilya
Ilya 2012-8-14

0 个投票

Other than a few typos (for example, y1 = (1:132)'>70; clearly is not what you meant), I don't see anything wrong with your code.
glmfit/glmval is your classifier.
The choice of the positive and negative classes is arbitrary. You can always swap them. By convention, the predicted scores must be large for the positive class and small for the negative class. For example, for the first ROC you choose ecan+ as your positive class. Then, I suppose, you set y1 to 1 for ecan+ and 0 for can-. The response returned by your logistic regression model will then be high for ecan+ and low for can-.
You can find many sources (textbooks, web pages etc) that interpret FPR, TPR and AUC.

5 个评论
显示 3更早的评论 隐藏 3更早的评论

Yasmin Tamimi
Yasmin Tamimi 2012-8-14
编辑：Yasmin Tamimi 2012-8-14
I really appreciate your help and still have some questions. First, why did you think I didn't mean it when I wrote y1 = (1:132)'>70,, I have 70 samples from can-, 62 from ecan+ and 12 from dcan+!! do you think I have to pad them with nans to get them all the same dimensions?! Second, when I say that AUC1 = 0.7191 what does that mean?! and what happens or what physical meaning does it give if I only considered like one feature each time from the 16 - I mean at that time I have to repeat the algorithm 3x16- ?! Finally, what does it mean response returned by your logistic regression model will then be high for ecan+ and low for can- ,, does it mean that the features in ecan+ are more significant than can- ?!!
Ilya
Ilya 2012-8-14
编辑：Ilya 2012-8-14
Sorry, I didn't look at your code carefully. Your y1 is fine.
The response for the logistic regression at any point is a number between 0 and 1. If it is close to 1, it means ecan+ is more likely; if it is close to 0, it means can- is more likely.
You used all predictors (features) in data in your analysis. Your conclusions, whatever they are, are only valid for all these features included. If you want to drop some of your features, you would have to refit your 3 models and repeat the ROC analysis.
Yasmin Tamimi
Yasmin Tamimi 2012-8-15
Ok! now I understood. One final question: Instead of writing y1 in a binary form I tried writing it as a combination of -1 and I ended up with an error bcz in glm I wrote binomial!! so what kind of distribution do you suggest for now!!
Yasmin Tamimi
Yasmin Tamimi 2012-8-15
i used normal distribution and it gave me the same results as in binomial!! no difference!!
Ilya
Ilya 2012-8-15
Your response values are categorical, which means your problem can be treated as a classification problem. The binomial distribution for glmfit is then the obvious choice.
Statistics Tlbx provides many other tools for classification. You might want to start here:
http://www.mathworks.com/help/toolbox/stats/f15360dfi14.html
http://www.mathworks.com/help/toolbox/stats/bsvjxt5.html
There is no way to predict what model would work best for you. "Best" is usually chosen by balancing the model accuracy and its interpretability.

请先登录，再进行评论。

更多回答（1 个）

Ilya
Ilya 2012-8-14

0 个投票

This is a rather vague question. You might get more replies if you explain what approaches you are thinking about and what you have tried so far.
The three groups of people define the labels. You must choose one of these groups as your positive class. Scores typically represent model predictions. You could train a classifier, for instance, to separate the 3 classes and use its predictions.
A ROC curve is used for two classes (groups). I don't know what you mean when you say you need AUC for 3 groups.

1 个评论
显示 -1更早的评论 隐藏 -1更早的评论

Yasmin Tamimi
Yasmin Tamimi 2012-8-14
i have chosen combinations of two from the three groups,, so i did three different cases!! But what do u mean that I have to train my classifier!! I updated my question and added my code..

请先登录，再进行评论。

类别

在 帮助中心 和 File Exchange 中查找有关 Discriminant Analysis 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by