PLOTRELIABILITY(TARGETS, PREDICTIONS) uses TARGETS and PREDICTIONS vectors to compute TARGETS_MEANS and PREDICTIONS_MEANS. FIG is a rudimentary reliability diagram plotted using these means.
Values of TARGETS are either 0 or 1, representing the two classes. Values of PREDICTIONS are probabilities of the positive class (1), previously obtained using a probabilistic predictive model. TARGETS and PREDICTIONS are row or column vectors of the same size.
The predictions are discretized into ten bins. Cases with predicted value between 0 and 0.1 fall in the first bin, between 0.1 and 0.2 in the second bin, etc. For each bin, the mean predicted value (PREDICTIONS_MEANS) and the true fraction of positive cases (TARGETS_MEANS) are computed. These points are then plotted on the X and Y axes respectively. If the model is well calibrated, the points will fall near the diagonal line.
SHOW_FIG is an optional logical input argument. Its default value is true. Setting it to false prevents the automatic display of the reliability diagram.
Syntax:
plotreliability(targets, predictions);
[fig, targets_means, predictions_means] = plotreliability(targets, predictions, false);
Remarks:
This function does not allow the number of bins or the width of each bin to be changed.
References:
Alexandru Niculescu-Mizil and Rich Caruana (2005) Predicting Good Probabilities With Supervised Learning, in Proceedings of the 22nd International Conference on Machine Learning. See section 4 (Qualitative Analysis of Predictions).
引用格式
Skynet (2024). Reliability Diagram for calibration of two-class predictor. (https://www.mathworks.com/matlabcentral/fileexchange/25704-reliability-diagram-for-calibration-of-two-class-predictor), MATLAB Central File Exchange. 检索来源 .
MATLAB 版本兼容性
平台兼容性
Windows macOS Linux类别
标签
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!版本 | 已发布 | 发行说明 | |
---|---|---|---|
1.3.0.0 | Fixed error in Description. |
||
1.2.0.0 | (1) Changed function name to plotreliability.
|
||
1.1.0.0 | (1) Added SHOW_FIG, an optional logical input argument. Its default value is false. Setting it to true displays the reliability diagram.
|
||
1.0.0.0 |