The issue you are encountering is a common LSTM issue which can arise due to multiple factors. LSTMs can take some time to learn the context of a sequence. Since the ‘XTrain2’ is from a different environment, the model was not quite accurate while predicting the first value.
The following strategies can help resolve the issue:
1. Data Normalisation: The code provided uses “zscore” function for normalisation. Make sure that the data in ‘XTrain2’ is normalized in the same way as ‘XTrain’ beacuse differences in data scaling can lead to poor predictions, especially at the start of a sequence.
If ‘XTrain’ and ‘XTrain2’ have different distributions, the statistics (mean and standard deviation) from ‘XTrain’ should be used to normalize ‘XTrain2’.
You can also make use of Layer Normalisation and Batch Normalisation in MATLAB. Refer to the following documentation links for more details on these layers:
“LayerNormalizationLayer”: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.layernormalizationlayer.html
2. Model Regularization: Adding dropout layers or L2 regularization helps the model generalize better and reduce overfitting to the training data environment.
Refer to the following documentation links for more details on these layers:
“setL2Factor”: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.layer.setl2factor.html
I hope this helps resolve the issue.
