How to caculate the size of input weight matrix when using the cell data to train a neural network?

3 次查看（过去 30 天）
子桥 陈
子桥 陈 2023-5-22
回答： Hari Raja 2023-11-21
Hello everyone,
recently I am learning to use NARX for time series modeling and forecasting. Now I have doubts about the number of input weights for training the neural network with cell arrays.
By reading previous questions, I successfully used multiple sets of time series to train a single model, that is, using cell structure data. The problem URL is here.
Simply put, for an N-input-single-output model, if I want to use M sets of input data with different time series lengths (up to T time units) for training, you can build a 1 × T cell array, in each cell Contains an N × M double array. Use NaN to fill data gaps due to different time series lengths.
Taking my work as an example, it contains 4 inputs and 1 output. I selected 10 sets of data to build a training set, and the longest time length is 482. So the size of my cell array is 1 × 482, and each cell is a 4 × 10 double array.
Set the hidden layer of NARX to 1, the number of neurons to 10, and the time delay to 2.
clc;clear;close all;
load('E:\Engine\up_N1pre.mat')
X = Input_train; % 1 × 482 cell, each cell is 4 × 10 double
T = N1_train; % 1 × 482 cell, each cell is 1 × 10 double
trainFcn = 'trainlm';
inputDelays = 1:2;
feedbackDelays = 1:2;
hiddenLayerSize = 10;
net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);
[x,xi,ai,t] = preparets(net,X,{},T);
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Train the Network
[net,tr] = train(net,x,t,xi,ai);
After successfully training the open-loop NARX network, I checked the input weights using net.IW. The structure is as follows:
2×2 cell array
{10×8 double} {10×2 double}
{ 0×0 double} { 0×0 double}
I can understand that net.IW{1,1} is related to input, and net.IW{1,2} is related to exogenous input. In terms of generalization, I think that if the time delay is modified to K for training in this example, the size of net.IW obtained after training is as follows:
2×2 cell array
{10×4×K double} {10×K double}
{ 0×0 double} { 0×0 double}
For comparison, I used the BP neural network to perform network training on the same input training set, setting 1 hidden layer and 10 neurons:
clc;clear;close all;
load('E:\Engine\up_N1pre.mat')
x_train_data=Input_train; % 1 × 482 cell, each cell is 4 × 10 double
y_train_data=N1_train; % 1 × 482 cell, each cell is 1 × 10 double
net=newff(x_train_data,y_train_data,10,{'tansig','purelin'});
[net,~]=train(net,x_train_data,y_train_data);
The resulting net.IW structure is as follows:
2×1 cell array
{10×8 double}
{0×0 double}
I am very confused here, why is net.IW{1,1} not a 10×4 double array? Is it because of the special impact of the cell array as a training set.I assume that the connection structure between network input and hidden layers is shown in the following figure：
To sum up, I have two main questions:
1. Is the number of input weights for NARX network training the same as I think?
2. Why is the size of the input weight matrix doubled when training the BP network?
Thanks in advance.
Chen

请先登录，再回答此问题。

回答（2 个）

Neha
Neha 2023-5-31
Hi Chen,
I understand that you want to find out the size of each input weight for the given two examples of NARX and BP network. I have slightly modified your code to change the values of input delays and feedback delays.
Xtrain={}
Ytrain={}
trainFcn = 'trainlm';
for i=1:50
Xtrain{i}=randn(5,10);
end
for i=1:50
Ytrain{i}=randn(1,10);
end
inputDelays = 1:3;
feedbackDelays = 1:6;
hiddenLayerSize = 10;
net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize,'open',trainFcn);
[x,xi,ai,t] = preparets(net,Xtrain,{},Ytrain);
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Train the Network
[net,tr] = train(net,x,t,xi,ai);
Input delay is the number of timesteps by which the input is delayed, and feedback delay is the number of timesteps by which the output is delayed.
>> net.IW
ans =
2×2 cell array
{10×15 double} {10×6 double}
{ 0×0 double} { 0×0 double}
Here, size(net.IW{1,1})= 10x15 which is essentially hiddenLayerSize x (inputDelays * Number of inputs)
size(net.IW{1,2})=10x6 which is essentially hiddenLayerSize x feedbackDelays
Regarding the BP network, I found that your assumption is correct. From the following code it can be found that:
size(bnet.IW{1,1})=10x5 which is essentially hiddenLayerSize x Number of inputs
Xtrain={}
Ytrain={}
trainFcn = 'trainlm';
for i=1:50
Xtrain{i}=randn(5,10);
end
for i=1:50
Ytrain{i}=randn(1,10);
end
bnet=newff(Xtrain,Ytrain,10,{'tansig','purelin'});
[net,~]=train(bnet,Xtrain,Ytrain);
Hope this helps!
Hari Raja
Hari Raja 2023-11-21
% Load data from the file
data = load('alphabet.mat');
inputs = data.inputs; % Assuming 'inputs' contains 9x7 matrices representing alphabets
% Reshape input data
X = reshape(inputs, [], 9*7)';
% Define the neural network architecture
net = feedforwardnet([50]); % Single hidden layer with 50 neurons
net.trainParam.epochs = 100;
net.trainParam.lr = 0.01; % Set your learning rate
net.trainParam.mc = 0.9; % Set your momentum factor
% Set the target output same as input (autoencoder)
targets = X;
% Train the neural network
net = train(net, X, targets);
% Test the trained network
outputs = sim(net, X);
% Display original and reconstructed data
figure;
subplot(1, 2, 1);
imshow(inputs(:,:,1), 'InitialMagnification', 'fit');
title('Original Data');
subplot(1, 2, 2);
imshow(reshape(outputs(:, 1), [9, 7]), 'InitialMagnification', 'fit');
title('Reconstructed Data');
% Check performance with different learning rates and momentum factors
learning_rates = [0.1, 0.01, 0.001];
momentum_factors = [0.9, 0.5, 0.1];
for lr = learning_rates
for mc = momentum_factors
net.trainParam.lr = lr;
net.trainParam.mc = mc;
% Retrain the network
net = train(net, X, targets);
% Evaluate performance or visualize results as needed
% ...
end
end

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by