Equal outputs using customized regression output layer

1 次查看（过去 30 天）
Fabrizio Bernardi
Fabrizio Bernardi 2020-10-11
回答： Shashank Gupta 2020-10-14
Hello everyone. I defined a customized regression output layer using the provided template and used the MSE for the Loss function:
classdef mseRegressionLayer < nnet.layer.RegressionLayer
methods
function layer = mseRegressionLayer(name)
% Set layer name.
layer.Name = name;
% Set layer description.
layer.Description = 'Mean square error';
end
function loss = forwardLoss(layer, Y, T)
% the predictions Y and the training targets T.
loss= mse(Y,T,'DataFormat','T')/size(Y,2);
end
end
end
Then I used this script to a trial dataset of arrays, the same one that has been trained in the example of training shallow neural networks here https://it.mathworks.com/help/deeplearning/ug/train-and-apply-multilayer-neural-networks.html , but now I wanted to build myself the Loss function, so I tried with this:
layer = mseRegressionLayer('mse');
load bodyfat_dataset % dataset with bodyfatInputs and bodyfatTargets
layers = [
sequenceInputLayer(13)
lstmLayer(100,'OutputMode',"last") % Output the last time step of the sequence
fullyConnectedLayer(1)
mseRegressionLayer('mse')];
%options = trainingOptions('sgdm');
options = trainingOptions('adam', ...
'MaxEpochs',100, ...
'GradientThreshold',1, ...
'InitialLearnRate',0.001, ...
'LearnRateSchedule','piecewise', ...
'LearnRateDropPeriod',100, ...
'LearnRateDropFactor',0.1,...
'Verbose',1, ...
'Plots','training-progress');
N = 100; %number of sequences
cellArrTrain = cell(N,1);
for i = 1:N
seq = bodyfatInputs(:,i);
seq = num2cell(seq,1);
cellArrTrain(i) = seq;
end
net = trainNetwork(cellArrTrain,bodyfatTargets(1:N)',layers,options);
YPred = predict(net,cellArrTrain);
predictionError = YPred - bodyfatTargets;
The problem is that the outputs in this way are all basically the same number... I don't think that is caused by the particular loss function because also using the mae error, on the standard template provided in the documentation, it does the same. The mini batch Loss acceptable, but outputs are too similiar. What can be the reason for it?
Thank you for your help!

请先登录，再回答此问题。

回答（1 个）

Shashank Gupta
Shashank Gupta 2020-10-14
Hi Fabrizio,
On the first look, the code does look good and it should work, did you check increasing the number of epoch and decreasing the learning rate decay, it seems like, it should converge but learning rate decay is super high which is making loss to converge at higher value than expected and the model is not able to learn. Try removing the decay and check if the training is smooth. I can't find any flag anywhere in the code atleast at the first look.
I hope my suggestion helps you out, Update me if you find anything interesting.
Cheers.

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by