im new to running ML models in matlab so please try to explain it in simple words,i am attempting to run a tcn model on a dataset which has 8 features which includes the timestamps necessary as well. when attempting to train the model i get the following error when i attempt to reshape the data: "The training sequences are of feature dimension 2795 10 but the input layer expects sequences of feature dimension 9."
here is the code:
clc;
clear;
close all;
trainData = readtable('train_net_gen.csv');
testData = readtable('test_net_gen.csv');
trainData.Timestamp = datetime(trainData.Timestamp, 'InputFormat', 'yyyy-MM-dd HH:mm:ss');
testData.Timestamp = datetime(testData.Timestamp, 'InputFormat', 'yyyy-MM-dd HH:mm:ss');
trainData.month = month(trainData.Timestamp);
trainData.day_of_month = day(trainData.Timestamp);
trainData.hour = hour(trainData.Timestamp);
trainData.day_of_week = weekday(trainData.Timestamp) - 1;
trainData.minute_of_day = trainData.hour * 60 + minute(trainData.Timestamp);
testData.month = month(testData.Timestamp);
testData.day_of_month = day(testData.Timestamp);
testData.hour = hour(testData.Timestamp);
testData.day_of_week = weekday(testData.Timestamp) - 1;
testData.minute_of_day = testData.hour * 60 + minute(testData.Timestamp);
% Moving averages and differences
trainData.moving_avg_3_production = movmean(trainData.next_day_from_system_kwh, [2, 0]);
trainData.diff_production = [NaN; diff(trainData.next_day_from_system_kwh)];
testData.moving_avg_3_production = movmean(testData.next_day_from_system_kwh, [2, 0]);
testData.diff_production = [NaN; diff(testData.next_day_from_system_kwh)];
% Remove NaNs
trainData = rmmissing(trainData);
testData = rmmissing(testData);
% Prepare features and target variables
sequenceLength = 10;
featureDimension = 9;
X_train = [trainData.minute_of_day, trainData.from_grid_kwh, trainData.net_consumption, ...
trainData.month, trainData.day_of_month, trainData.hour, trainData.day_of_week, ...
trainData.moving_avg_3_production, trainData.diff_production];
y_train = trainData.next_day_from_system_kwh;
X_test = [testData.minute_of_day, testData.from_grid_kwh, testData.net_consumption, ...
testData.month, testData.day_of_month, testData.hour, testData.day_of_week, ...
testData.moving_avg_3_production, testData.diff_production];
y_test = testData.next_day_from_system_kwh;
% Standardize features
X_train = (X_train - mean(X_train)) ./ std(X_train);
y_train = (y_train - mean(y_train)) ./ std(y_train);
X_test = (X_test - mean(X_test)) ./ std(X_test);
y_test = (y_test - mean(y_test)) ./ std(y_test);
% Calculate the exact number of samples to ensure all data is used
numTrainSamples = floor(size(X_train, 1) / sequenceLength);
numTestSamples = floor(size(X_test, 1) / sequenceLength);
% Reshape to ensure correct input dimensions: [numSamples, sequenceLength, featureDimension]
X_train_seq = reshape(X_train(1:numTrainSamples * sequenceLength, :), [numTrainSamples, sequenceLength, featureDimension]);
y_train_seq = y_train(1:numTrainSamples);
X_test_seq = reshape(X_test(1:numTestSamples * sequenceLength, :), [numTestSamples, sequenceLength, featureDimension]);
y_test_seq = y_test(1:numTestSamples);
% Verify the dimensions
fprintf('Size of X_train_seq: %s\n', mat2str(size(X_train_seq)));
fprintf('Size of y_train_seq: %s\n', mat2str(size(y_train_seq)));
layers = [
sequenceInputLayer(featureDimension, "Name", "input")
convolution1dLayer(3, 64, "Padding", "causal", "Name", "conv1")
layerNormalizationLayer("Name", "layernorm1")
reluLayer("Name", "relu1")
convolution1dLayer(3, 128, "Padding", "causal", "Name", "conv2")
layerNormalizationLayer("Name", "layernorm2")
reluLayer("Name", "relu2")
convolution1dLayer(3, 128, "Padding", "causal", "Name", "conv3")
layerNormalizationLayer("Name", "layernorm3")
reluLayer("Name", "relu3")
fullyConnectedLayer(1, "Name", "output")
regressionLayer("Name", "outputLayer")
];
options = trainingOptions("adam", ...
"MaxEpochs", 100, ...
"MiniBatchSize", 16, ...
"Shuffle", "never", ...
"ValidationData", {X_test_seq, y_test_seq}, ...
"Plots", "training-progress", ...
"Verbose", 1);
try
net = trainNetwork(X_train_seq, y_train_seq, layers, options);
disp("Training successful with simplified sequence-to-one TCN structure!");
catch ME
disp("Error during training:");
disp(ME.message);
end
if exist("net", "var")
y_pred = predict(net, X_test_seq);
rmse = sqrt(mean((y_pred - y_test_seq).^2));
disp(["Production RMSE: ", num2str(rmse)]);
figure;
plot(y_test_seq, "b-", "LineWidth", 1.5);
hold on;
plot(y_pred, "r--", "LineWidth", 1.5);
xlabel("Sample");
ylabel("Standardized kWh");
legend("Actual", "Predicted");
grid on;
end
output:
Size of X_train_seq: [2795 10 9]
Size of y_train_seq: [2795 1]
Error during training:
The training sequences are of feature dimension 2795 10 but the input layer expects sequences of feature dimension 9.
I would like any suggestions on why this error occurs as even when i use it for a subset of features such as 5 features, it still reads the feature dimensions as 10.
Thank you.
