Hi,
To build a critic with RNN consider using the below code snippet that uses “sequenceInputLayer” connected to “lstmLayer” which is then connected to "fullyConnectedLayer" and “relu” as shown below :
obsInfo = rlNumericSpec([5 1320000]);
actInfo = rlFiniteSetSpec([0, 1]);
% Create the environment
env = rlFunctionEnv(obsInfo, actInfo, @myStepFunction, @myResetFunction);
% Define the critic network with an LSTM layer
dnn = [
sequenceInputLayer([5 1320000],'Normalization','none','Name','obsInfo')
lstmLayer(24,'OutputMode','sequence','Name','CriticLSTM')
fullyConnectedLayer(24,'Name','CriticStateFC1')
reluLayer('Name','CriticRelu1')
fullyConnectedLayer(24, 'Name','CriticStateFC2')
reluLayer('Name','CriticCommonRelu')
fullyConnectedLayer(length(actInfo.Elements),'Name','output')];
% Define critic options
criticOpts = rlRepresentationOptions('LearnRate', 0.001, 'GradientThreshold', 1);
% Create the critic representation
critic = rlQValueRepresentation(dnn, obsInfo, actInfo, 'Observation', {'obsInfo'}, criticOpts);
Please refer to the following documentation on how to build a network on sequential input:
