As described above, I would suggest you to perform k-Fold Cross Validation. K-Fold Cross Validation helps in splitting the data into K-blocks, training the Model on "K-1" folds and testing the trained model on the left out Kth fold ( to report the generalization error). You should be performing the following steps : -
- Split data into train and test sets.
- K-Fold cross-validate on the training data to estimate generalization error.
- Select the model with least generalization error.
- Train it from scratch with the entire training data
- Test it on the test data.
These steps can help you to automate the process.
You can go through the following link to know more about cross-validation and ease of coding your model.
