Hi,
I believe there is a limit of 20 classes for labels to be displayed in the confusion matrix. There is an existing enhancement request in place to clarify this in our documentation. I have updated this ticket reflect your pain point as well.
To answer your second question, you can export your model from the app and then use its "predictFcn" to evaluate its response for the training data. You can then use this result along with the true class values as inputs for the "confusionmat" function to retrieve the confusion matrix.
confusionmat(Y_actual,trainedModel.predictFcn(TrainingData__as_used_in_app))
Please note that the results will be slightly different from what you see in the Classification Learner App because the app uses 5-fold cross-validation by default. The validation scheme only affects the way that Classification Learner computes validation metrics. The final exported model is always trained using the full data set.
