error message: 'dlgradient' inputs must be traced dlarray objects or cell arrays, structures or tables containing traced dlarray objects. To enable tracing, use 'dlfeval'.

Question

MA 2021-12-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1609780-error-message-dlgradient-inputs-must-be-traced-dlarray-objects-or-cell-arrays-structures-or-tabl

编辑： Kingshuk 2024-2-18

I want a small network that contains only a batch normalization layer followed by fully-connected layer, softmax and finally cross entropy loss. To do that, my model is implemented as follows:

function [dlY1,state] = model(parameters,dlX,doTraining,state)
% Batch normalization, ReLU
    offset = parameters.batchnorm1.Offset;
    scale = parameters.batchnorm1.Scale;
    trainedMean = state.batchnorm1.TrainedMean;
    trainedVariance = state.batchnorm1.TrainedVariance;
    
    if doTraining
        [dlY,trainedMean,trainedVariance] = batchnorm(dlX,offset,scale,trainedMean,trainedVariance);
        
        % Update state
        state.batchnorm1.TrainedMean = trainedMean;
        state.batchnorm1.TrainedVariance = trainedVariance;
    else
        dlY = batchnorm(dlX,offset,scale,trainedMean,trainedVariance);
    end
    
    dlY = relu(dlY);
    
    % Fully connect, softmax (labels)
    weights = parameters.fc1.Weights;
    bias = parameters.fc1.Bias;
    dlY1 = fullyconnect(dlY,weights,bias);
    dlY1 = softmax(dlY1);
end

TO calculate the gradients, I have this function here:

function [gradients,state,loss] = modelGradients(parameters,dlX,T1,state)
    doTraining = true;
    [dlY1,state] = model(parameters,dlX,doTraining,state);
    
    lossLabels = crossentropy(dlY1,T1);
    
    loss = lossLabels;
    
    gradients = dlgradient(loss,parameters,'EnableHigherDerivatives',true);
end

TO start the program with simple example to calculate the derivatives:

 dlX=dlarray(rand(10,5),'BC');
T1=[1;2];  T1=repelem(T1,5);
dlY=onehotencode(categorical(T1'),1);
sz=[2,5];
numOut=2;
numIn=5;
parameters.batchnorm1.Offset = zeros([5 1]);
parameters.batchnorm1.Scale = ones([5 1]);
state.batchnorm1.TrainedMean = zeros(5,1,'single');
state.batchnorm1.TrainedVariance = ones(5,1,'single');
parameters.fc1.Weights=initializeGlorot(sz,numOut,numIn);
parameters.fc1.Bias=zeros(2,1);
[gradients,state,loss] = dlfeval(@modelGradients, parameters, dlX, dlY, state);

So, the gradients are the thing I am looking for and the output of this process is this following error message:

Error using dlfeval (line 43)
'dlgradient' inputs must be traced dlarray objects or cell arrays,
structures or tables containing traced dlarray objects. To enable tracing,
use 'dlfeval'.

I tried to use dlfeval instead of dlgradient but another error message appears:

Error using dlfeval (line 43)
Nested dlfeval calls are not supported. To compute higher derivatives, set
the 'EnableHigherDerivatives' option of the dlgradient function to true.

Does anyone got any idea why I recieve these error messages? THanks.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Kingshuk 2024-2-18

编辑：Kingshuk 2024-2-18

Hello sir, I got the same for to train my encoder decoder (contain LSTM layer) model.

How to solve the issue, if you help me please.

Error using dlarray/dlgradient

'dlgradient' inputs must be traced dlarray objects or cell arrays, structures or tables containing traced

dlarray objects. To enable tracing, use 'dlfeval'.

This is the error which is coming while training

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

MA 2021-12-13

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1609780-error-message-dlgradient-inputs-must-be-traced-dlarray-objects-or-cell-arrays-structures-or-tabl#answer_854015

在 MATLAB Online 中打开

It seems I solve it. The parameters needs to be dlarry as well as follows:

parameters.batchnorm1.Offset = initializeZeros([5 1]);
parameters.batchnorm1.Scale = initializeOnes([5 1]);
state.batchnorm1.TrainedMean = zeros(5,1,'single');
state.batchnorm1.TrainedVariance = ones(5,1,'single');
numClasses=2; 
parameters.fc1.Weights = initializeGlorot(sz,numOut,numIn);
parameters.fc1.Bias = initializeZeros([numClasses 1]);
[gradients,state,loss] = dlfeval(@modelGradients, parameters, dlX, dlY, state);