Custom loss function for DNN training

20 次查看（过去 30 天）
Fotios Mantoglou
Fotios Mantoglou 2022-7-12
编辑： David Ho 2023-10-11
I have the feed forward network as shown bellow. It consists of an input sequence layer, 2 hidden fully conected and an output regression layer.
Network architecture:
I have defined the layers as follows:
layers = [
sequenceInputLayer(4,"Name","sequence")
fullyConnectedLayer(4,"Name","fc_1")
fullyConnectedLayer(4,"Name","fc_2")
regressionLayer("Name","regressionoutput")];
I need to train the model where the input will be the same as the output. I need to restrict the weights of the layers using the following loss function:
L=L1+L2+L3+L4
where
L1= λ|cov(H)-I|
L2= λ|(H/4)+λ|W1*transpose(W1)-I|
L3= λ|cov(Q)-I|
L4= mse of input-output
H is the output of the 1st hidden layer and Q is the output of the second hidden layer and W the weight of each layer.The total loss function is L=L1+L2+L3+L4.
What is the best approach for this? Could posibly be done with regularisation?
Thank you!

请先登录，再回答此问题。

回答（2 个）

Pratyush Swain
Pratyush Swain 2023-10-10
Hi Fotios,
I understand you want to create a custom deep learning network.Please refer to https://in.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layers.html. It contains information regarding custom 'Regression output layer' and 'Intermediate layer'.
The output layer uses two functions to compute the loss and the derivatives: forwardLoss and backwardLoss. The forwardLoss function computes the loss L. The backwardLoss function computes the derivatives of the loss with respect to the predictions.
For example: To write a weighted cross entropy classification loss, try running this in the MATLAB command window
>> edit(fullfile(matlabroot,'examples','deeplearning_shared','main','weightedClassificationLayer.m'))
Hope this helps.
David Ho
David Ho 2023-10-11
编辑：David Ho 2023-10-11
Ran in:
Hello Fotios,
You can solve this constrained learning problem using a dlnetwork object and a custom training loop:
https://uk.mathworks.com/help/deeplearning/ug/train-network-using-custom-training-loop.html
If I understand your question correctly, I believe the loss function should look something like this:
layers = [
sequenceInputLayer(4,"Name","sequence")
fullyConnectedLayer(4,"Name","fc_1")
fullyConnectedLayer(4,"Name","fc_2")];
net = dlnetwork(layers);
% Test loss function with random inputs and targets
rng(0);
X = dlarray(rand(4,10), "CT");
T = dlarray(rand(4,10), "CT");
lambda = 1;
loss = dlfeval(@(net,X,T) modelLoss(net, X, T, lambda), net, X, T)
loss =
1×1 single dlarray 58.8300
grad = 4×3 table
Layer Parameter Value ______ _________ _____________ "fc_1" "Weights" {4×4 dlarray} "fc_1" "Bias" {4×1 dlarray} "fc_2" "Weights" {4×4 dlarray} "fc_2" "Bias" {4×1 dlarray}
function [loss, grad] = modelLoss(net,X,T,lambda)
% Get the activations and weights of the fully connected layers
[h,q] = forward(net, X, Outputs=["fc_1", "fc_2"]);
w1 = net.Layers(2).Weights;
% Compute covariance matrices
covH = cov(stripdims(h)');
covQ = cov(stripdims(q)');
% Compute components of loss function
l1 = dlNorm(lambda*covH - eye(size(covH)));
l2 = lambda*dlNorm(h/4 + lambda*norm(w1*w1' - eye(size(w1)), "fro"));
l3 = lambda*dlNorm(covQ - eye(size(covQ)));
l4 = mse(q,T);
% Compute loss and gradients
loss = l1 + l2 + l3 + l4;
grad = dlgradient(loss, net.Learnables);
end
function normX = dlNorm(X)
% Frobenius norm of a dlarray
normX = sum(stripdims(X)'*stripdims(X), "all");
normX = sqrt(normX);
end

类别

在 Help Center 和 File Exchange 中查找有关 Image Data Workflows 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by