fitcensemble with AdaBoostM2 is not a probabilistic model. It outputs classification scores (margins), not calibrated probabilities.The ScoreTransform option ('logit', 'doublelogit', etc.) just applies a monotone mapping to those scores.It does not normalize them into probabilities that sum to 1 across classes.That’s why your row sums are not equal to 1.
You can try the following steps:
1. You can use a learner/ensemble that supports posterior probabilities. For example:
mdl = fitcensemble(X, Y, 'Method', 'Bag');
[pred, prob] = predict(mdl, X);
2. Train your AdaBoost model, then fit a calibration model (e.g. logistic regression or isotonic regression) to map raw scores to probabilities.In MATLAB, this is available via:
mdl = fitcensemble(X, Y, 'Method','AdaBoostM2');
mdlCal = fitSVMPosterior(mdl); % or fitcecoc + fitPosterior
[pred, prob] = predict(mdlCal, X);
Though fitSVMPosterior works mainly for SVM, the idea is similar: you need a separate calibration step.
3. Wrap AdaBoost inside fitcecoc, then use fitPosterior to calibrate:
t = templateEnsemble('AdaBoostM2',100,'Tree');
mdl = fitcecoc(X, Y, 'Learners', t);
mdl = fitPosterior(mdl, X, Y);
[pred, prob] = predict(mdl, X);
