About Friedman test implementation

2012 2 21
1 个回答
回答已采纳
更新时间：2013 10 18
6 次查看（30 天）

0 个投票

Hi,
I found (maybe I am wrong) that the Matlab implementation of the Friedman test does not give me the same results than a "classical" hand-made resolution for a problem.
I have used the dataset of the Table 6 of the paper "Statistical Comparisons of Classifiers over Multiple Data Sets" (Demsar, Journal of Machine Learning, 2006).
Even the mean ranks of the different classifiers (columns) aren't the same. From that moment, chi-square value does not agree, etc...
Does anybody know why are these differences? Best regards.

2 个评论
显示 无 隐藏 无

the cyclist
the cyclist 2012-2-21
You might have a better chance of someone exploring this question if you pasted the data from that paper here, in a way that someone could just directly put it into MATLAB.
José Manuel
José Manuel 2012-2-21
You are right.
data=[0.7630, 0.7680, 0.7710, 0.7980; 0.5990, 0.5910, 0.5900, 0.5690; 0.9540, 0.9710, 0.9680, 0.9670; 0.6280, 0.6610, 0.6540, 0.6570; 0.8820, 0.8880, 0.8860, 0.8980; 0.9360, 0.9310, 0.9160, 0.9310; 0.6610, 0.6680, 0.6090, 0.6850; 0.5830, 0.5830, 0.5630, 0.6250; 0.7750, 0.8380, 0.8660, 0.8750; 1 , 1 , 1 , 1 ; 0.9400, 0.9620, 0.9650, 0.9620; 0.6190, 0.6660, 0.6140, 0.6690; 0.9720, 0.9810, 0.9750, 0.9750; 0.9570, 0.9780, 0.9460, 0.9700];
And I add this question:
Regarding to the "two categorical factors, what is the meaning of this phrase os the help? "Friedman’s test is a nonparametric test for data having a two-way layout (data grouped by two categorical factors)." I noticed that all the examples of the help are with only two factors in the row dimension (with several reps)...

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

the cyclist
the cyclist 2012-2-21

3 个投票

I notice a couple things. First, that paper is doing the ranking from largest to smallest effect. MATLAB is doing the opposite. This is just a matter of convention, and should not effect the final results. From MATLAB, I get
meanranks: [1.8571 3 2.0714 3.0714]
The paper gives [3.143 2.000 2.893 1.964].
If we "invert" the MATLAB result, it would be
[3.1429 2.0000 2.9286 1.9286].
These results are (almost) the same as the paper.
I'm not sure, but the remaining differences is likely due to how tied ranks are handled. My advice would be to open up the file ( edit friedman ), place a breakpoint at the beginning, and step through the code line-by-line to see how the exact ranks are calculated.

5 个评论
显示 3更早的评论 隐藏 3更早的评论

José Manuel
José Manuel 2012-2-21
OK, thank you very much.
How do you "invert" the MATLAB result?
the cyclist
the cyclist 2012-2-21
I did it by doing the Friedman test on the negative of the results in the table. So, I did friedman(-data) rather than friedman(data).
José Manuel
José Manuel 2012-2-21
Thank you very much again.
I was looking for any parameter to tell the friedman function the criterion to assing the ranks.
I have noticed that the small error between the means of the paper [3.143 2.000 2.893 1.964] and the means of Matlab [3.1429 2.0000 2.9286 1.9286] comes from an error in the paper, rankinkg the dataset on the penultimate row, because there is a draw that the authors have not detected. If one replicate the author's error, the result is the same.
And debugging, I also noticed that friedman function uses anova2 function, where the chi stat is calculated. So, I suppose that dissagreements in this value (9.28 in the paper vs 10.31 by Matlab) maybe come from some assumptions that are done (normality...) when actually Friedman test is non-parametric.
Do you agree?
the cyclist
the cyclist 2012-2-21
No, I do not agree. I believe that MATLAB is applying a correction term that is not in the paper. In the code for friedman() you'll see these lines:
% Get a matrix of ranks. For the unusual case of replicated
% measurements, rank together all replicates in the same row. This
% is the advice given by Zar (1996), "Biostatistical Analysis."
m = X;
sumta = 0;
for j=1:r
jrows = reps * (j-1) + (1:reps);
v = X(jrows,:);
[a,tieadj] = tiedrank(v(:));
m(jrows,:) = reshape(a, reps, c);;
sumta = sumta + 2*tieadj;
end
I am not familiar the reason this is necessary, but if you look at the calculation of chistat inside friedman(), you will see this correction applied when "sumta" is not equal to zero. That is what makes the MATLAB result different from the paper.
José Manuel
José Manuel 2012-2-22
You are right. I also do not know why Matlab does that.
Thank you very much.

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Analysis of Variance and Covariance 的更多信息

产品

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by