No information rate test
显示 更早的评论
Hi, I'm here to ask you if there exists something similar to the no information rate test in matlab, I want to explain myself better: during classification analysis I met the need to statistically compute the p-value of a function that allows me to test the hypothesis that the accuracy (true predicted label / true label) is actually better than no information rate (is the proportion of the largest class within the dataset)
回答（1 个）
Aneela
2024-3-1
0 个投票
Hi Alberto Azzari,
In MATLAB, there isn't a built-in function that directly computes the p-value to test whether the accuracy of a classifier is significantly better than the no information rate (NIR).
However, you can refer to the workflow below for Permutation Testing:
- Compute the No Information Rate (NIR) by finding the proportion of the largest class in your dataset.
- Compute the “observed accuracy” of the classifier on the original dataset.
- Randomly permute the class labels of your dataset many times, each time calculating the accuracy of your classifier with the permuted labels.
- This will give you a distribution of accuracies under Null Hypothesis.
- The p-value is the proportion of accuracies from the permutation test that are equal to or greater than the observed accuracy.
- A low p-value suggests that the observed accuracy is significantly better than NIR.
To prove that a model is significant, the accuracy should be higher than the NIR.
类别
在 帮助中心 和 File Exchange 中查找有关 Hypothesis Tests 的更多信息
2021-12-21
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
