How to conduct Friedman test to compare metaheuristics and which data sets I should consider?

5 次查看（过去 30 天）
Rasel Ahmed
Rasel Ahmed 2021-2-17
回答： Hornett 2024-9-27
Dear all,
I am working on metaheuristics. I want to do Friedman test to compare the algorithms. I run each algorithm 32 times and stored the best, mean, median values from each run.
To perform the Friedman test which values I should consider, the best value (last value) from 32 runs of each algorithm, or median or mean values? and please explain the reason for selection of this particle values.
Please share your view about this test.
Thanks

请先登录，再回答此问题。

回答（1 个）

Hornett
Hornett 2024-9-27
In practice, the mean or median values are more commonly used for statistical comparisons in algorithm performance because they provide a more comprehensive view of the algorithm's behavior over multiple runs. The best values can be influenced by outliers and might not represent the typical performance of the algorithms.
Here's why you might prefer mean or median over best values:
  • Reproducibility: Mean or median values are more reproducible than the best values. An algorithm that consistently performs well on average is generally more reliable than one that performs exceptionally well only occasionally.
  • Robustness: Median values, in particular, can provide a more robust comparison when there are outliers in the data, as they are less sensitive to extreme values.
  • Statistical Power: The Friedman test assumes that the measurements are independent and identically distributed. Mean or median values, by summarizing the runs, may better meet these assumptions, especially if there is variability in the performance of the algorithms across runs.
In summary, if your goal is to compare the overall performance and reliability of the algorithms, you should consider using the mean or median values for the Friedman test. If you are specifically interested in the best possible performance regardless of consistency, you might opt for the best values, keeping in mind the potential limitations of this approach.
Friedman's test: https://www.mathworks.com/help/stats/friedman.html
Hope it Helps!

类别

在 Help Center 和 File Exchange 中查找有关 Analysis of Variance and Covariance 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by