Hi,
According to the documentation on “dlfeval” :
You can use a “dlnetwork” object as a function argument to evaluate gradients for deep learning. To do this, you should perform the network's forward pass within the "fun" function handle.
To implement the custom loss in the above network, refer to the function “modelGradients” from the corrected code below. Compute “gradients = dlgradient(loss, net.Learnables);” and then use “sgdmupdate(net, ...)” to update the network.
numObservations = 100;
numFeatures = 3;
numResponses = 3;
sequenceLength = 1;
X_train = rand(numFeatures, sequenceLength, numObservations);
Y_train = rand(numResponses, sequenceLength, numObservations);
numHiddenUnits = 100;
layers = [ ...
sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits, 'OutputMode', 'sequence')
fullyConnectedLayer(numResponses)
];
maxEpochs = 50;
miniBatchSize = 3;
learningRate = 0.01;
net = dlnetwork(layers);
numMiniBatches = floor(numObservations / miniBatchSize);
averageGrad = [];
averageSqGrad = [];
iteration = 0;
Velocities = [];
momentum = 0.9;
for epoch = 1:maxEpochs
idx = randperm(numObservations);
X_train_shuffled = X_train(:, :, idx);
Y_train_shuffled = Y_train(:, :, idx);
totalLoss = 0;
for i = 1:numMiniBatches
idxBatch = (i - 1) * miniBatchSize + 1 : i * miniBatchSize;
X_batch = X_train_shuffled(:, :, idxBatch);
Y_batch = Y_train_shuffled(:, :, idxBatch);
iteration = iteration + 1;
dlX_batch = dlarray(X_batch, 'CBT');
dlY_batch = dlarray(Y_batch, 'CBT');
[loss, gradients] = dlfeval(@modelGradients, net, dlX_batch, dlY_batch);
[net, Velocities] = sgdmupdate(net, gradients, Velocities, learningRate, momentum);
totalLoss = totalLoss + loss;
end
avgLoss = totalLoss / numMiniBatches;
fprintf('Epoch %d, Average Loss: %.4f\n', epoch, avgLoss);
end
function [loss, gradients] = modelGradients(net, dlX, dlY)
dlYPred = forward(net, dlX);
loss = mean((dlYPred - dlY).^2, 'all');
gradients = dlgradient(loss, net.Learnables);
end
Additionally, please refer to this example on how to train a network using custom loss :
