accuracy test higher than training

Question

soepblik 2021-11-9

回答： Sulaymon Eshkabilov 2021-11-10

Hi,

i have the following inputs to the classification learner:

label 'A' has 191x15 data inputs

label 'B' has 339x15 data inputs

What i now did is take 75% of both label A and label B as trainings data

and the rest is test data.

the strange thing is that i see that with cross validation 5 folds the accuracy is about 71% for for example ensemble and the test accuracy is 76%.

What is the problem? because I think the test accuracy must be lower than the validation accuracy.

What is the best way to fix this?

thanks in advance!

Answer 1

Sulaymon Eshkabilov 2021-11-10

You can try to employ random selection of the data for training and validation using randperm() or randsample(), for example.

Another approach to address the issue in general is to change the ratio of data for training and validation purposes, e.g. 65% vs. 35%.