The dimensions keep changing so the for loop is failing. How to address this?

Hi @Manny,

When working with LSTM networks in MATLAB, especially for time series forecasting, it is crucial to ensure that the dimensions of the input data are consistent throughout the training and prediction processes. The errors you are experiencing can be attributed to a few common pitfalls. Let me break down the issues and provide solutions step-by-step.

Concatenation Error: The error message Dimensions of arrays being concatenated are not consistent indicates that the dimensions of sTrain and yPred(t-1) do not match when you attempt to concatenate them. This typically occurs when the dimensions of the arrays differ in either the number of rows or columns.

Training Network Error: The error about training sequences are of feature dimension 3251 but the input layer expects sequences of feature dimension 1 suggests that the input data being fed into the network does not match the expected input size defined in the sequenceInputLayer. This mismatch can occur if the data is not reshaped correctly.

So, to resolve these issues, follow these steps:

Step 1: Ensure Consistent Dimensions

When concatenating sTrain and yPred(t-1), ensure that both arrays have the same number of rows. Since sTrain is a column vector, you should ensure that yPred(t-1) is also a column vector. You can achieve this by using the following code:

sTrain = [sTrain; yPred(t-1)]; % Use semicolon to concatenate vertically

Step 2: Reshape Data for Training

Before training the network, ensure that xTrain and yTrain are reshaped correctly. Since you are using a sequence input layer expecting a feature dimension of 1, you should reshape your training data accordingly:

   %   Reshape to [1, sequenceLength, numFeatures]
   xTrain = reshape(sTrainNorm(1:end-1), [1, numel(sTrainNorm(1:end-1)), 1]); 

   % Reshape similarly
  yTrain = reshape(sTrainNorm(2:end), [1, numel(sTrainNorm(2:end)), 1]); 

Step 3: Update the Prediction Loop

In your prediction loop, ensure that the new sTrain is updated correctly after each prediction. The updated code should look like this:

for t = 2:numForecasts
  sTrain = [sTrain; yPred(t-1)]; % Ensure vertical concatenation
  mu = mean(sTrain);
  sigma = std(sTrain);
  sTrainNorm = (sTrain - mu) / sigma;

    % Reshape for training
    xTrain = reshape(sTrainNorm(1:end-1), [1, numel(sTrainNorm(1:end-1)), 1]);
    yTrain = reshape(sTrainNorm(2:end), [1, numel(sTrainNorm(2:end)), 1]);

    net = trainNetwork(xTrain, yTrain, layers, options);
    yPred(t) = predict(net, reshape(yPred(t-1), [1, 1, 1])); % Reshape for prediction
  end

Here is the revised section of your code with the suggested changes:

for t = 2:numForecasts
  sTrain = [sTrain; yPred(t-1)]; % Concatenate vertically
  mu = mean(sTrain);
  sigma = std(sTrain);
  sTrainNorm = (sTrain - mu) / sigma;

    % Reshape for training
    xTrain = reshape(sTrainNorm(1:end-1), [1, numel(sTrainNorm(1:end-1)), 1]);
    yTrain = reshape(sTrainNorm(2:end), [1, numel(sTrainNorm(2:end)), 1]);

    net = trainNetwork(xTrain, yTrain, layers, options);
    yPred(t) = predict(net, reshape(yPred(t-1), [1, 1, 1])); % Reshape for prediction
  end

By ensuring that the dimensions of your arrays are consistent and reshaping your data appropriately, you should be able to resolve the errors you are encountering. MATLAB's LSTM implementation requires careful attention to the shape of the data, especially when dealing with sequences. If you continue to face issues, consider using debugging techniques such as checking the size of your variables at each step to identify where the mismatch occurs.

Hope this helps.