Why is dlgradient giving different answers?

1 次查看（过去 30 天）
Vellapandi M Research Scholar
回答： Angelo Yeo 2023-12-18
When I use the dlgradient function to compute the gradient of the expression (Parameters.fc2.Weights * tanh(Parameters.fc1.Weights * y(:,1) + Parameters.fc1.Bias) + Parameters.fc2.Bias) with respect to Parameters.fc2.Bias, it yields varying results instead of a consistent value of 1. According to theoretical calculations, it should be 1, but for different values of y(:,i), I observe discrepancies. What might be the issue?
Parameters = struct;
stateSize = 1;
hiddenSize = 20;
Parameters.fc1 = struct;
sz_fc1 = [hiddenSize stateSize];
Parameters.fc1.Weights = initializeGlorot(sz_fc1, hiddenSize, stateSize);
Parameters.fc1.Bias = initializeZeros([hiddenSize 1]);
Parameters.fc2 = struct;
sz_fc2 = [stateSize hiddenSize];
Parameters.fc2.Weights = initializeGlorot(sz_fc2, stateSize, hiddenSize);
Parameters.fc2.Bias = initializeZeros([stateSize 1]);
y(:,1) = 1;
y(:,2) = 0.976;
gradient1.fc2.Bias = dlgradient(Parameters.fc2.Weights * (tanh(Parameters.fc1.Weights * y(:,1) + Parameters.fc1.Bias)) + Parameters.fc2.Bias, Parameters.fc2.Bias)
gradient2.fc2.Bias = dlgradient(Parameters.fc2.Weights * (tanh(Parameters.fc1.Weights * y(:,2) + Parameters.fc1.Bias)) + Parameters.fc2.Bias, Parameters.fc2.Bias)

请先登录，再回答此问题。

采纳的回答

Angelo Yeo
Angelo Yeo 2023-12-18
Ran in:
You can try to incorporate dlfeval when using dlgradient. You can get the results of 1's as expected.
Parameters = struct;
stateSize = 1;
hiddenSize = 20;
Parameters.fc1 = struct;
sz_fc1 = [hiddenSize stateSize];
Parameters.fc1.Weights = initializeGlorot(sz_fc1, hiddenSize, stateSize);
Parameters.fc1.Bias = initializeZeros([hiddenSize 1]);
Parameters.fc2 = struct;
sz_fc2 = [stateSize hiddenSize];
Parameters.fc2.Weights = initializeGlorot(sz_fc2, stateSize, hiddenSize);
Parameters.fc2.Bias = initializeZeros([stateSize 1]);
y(:,1) = 1;
y(:,2) = 0.976;
[res1, res2] = dlfeval(@gradFun, Parameters, y)
res1 =
1×1 single dlarray 1
res2 =
1×1 single dlarray 1
function [res1, res2] = gradFun(Parameters, y)
res1 = dlgradient(Parameters.fc2.Weights * (tanh(Parameters.fc1.Weights * y(:,1) + Parameters.fc1.Bias)) + Parameters.fc2.Bias, Parameters.fc2.Bias);
res2 = dlgradient(Parameters.fc2.Weights * (tanh(Parameters.fc1.Weights * y(:,2) + Parameters.fc1.Bias)) + Parameters.fc2.Bias, Parameters.fc2.Bias);
end
function weights = initializeGlorot(sz,numOut,numIn)
Z = 2*rand(sz,'single') - 1;
bound = sqrt(6 / (numIn + numOut));
weights = bound * Z;
weights = dlarray(weights);
end
function parameter = initializeZeros(sz)
parameter = zeros(sz,'single');
parameter = dlarray(parameter);
end

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Dimensionality Reduction and Feature Extraction 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by