Hi Jason Butler,
To set an initial guess for the mean in PPO actor network, modify the initial weights or biases of the layers that contribute to calculating the mean.
- Set an initial guess for the mean in the bias of the “fullyConnectedLayer” in the mean calculation.
- However, because of the non-linearities like the “tanhLayer”, directly setting the bias to achieve a specific mean after scaling and non-linear transformations can be complex.
Assuming the desired initial mean as 5, here’s a workaround:
desiredInitialMean = 5; % Adjust this value as needed
% Since you have 3 actions, create a bias vector with 3 elements
biasForDesiredMean = repmat(desiredInitialMean / actInfo.UpperLimit, [prod(actInfo.Dimension), 1]);
% Modify the meanPath definition to include the bias initialization as a vector
meanPath = [
tanhLayer(Name="tanhMean");
fullyConnectedLayer(prod(actInfo.Dimension), ...
'Bias', biasForDesiredMean, ...
Name="fcMean");
scalingLayer(Name="scale", ...
'Scale', actInfo.UpperLimit)
];
For more information on “Bias” in the “fullyConnectedLayer”, refer to the following MathWorks documentation: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.fullyconnectedlayer.html?s_tid=doc_ta#:~:text=Layer%20biases%2C%20specified,single%20%7C%20double
