rlDiscrete​Categorica​lActor not accepting a mix of rlNumericSpec and rlFiniteSetSpec objects - observation for a RL environment

1 次查看（过去 30 天）
Pruthwiraj Santhosh
评论： Pruthwiraj Santhosh 2024-2-15
I am looking for an example of which implements a mix of rlNumericSpec and rlFiniteSetSpec object in an RL environment (as mentioned here). Some of my observations are numerical/continuous whereas others are finite/discrete.
I created a set of obervations which is mixture of rlNumericSpec and rlFiniteSetSpec objects using the following code:
obsInfo_numeric = rlNumericSpec([4 1]);
obsInfo_finite = rlFiniteSetSpec([1 1]);
obsInfo = [obsInfo_numeric,obsInfo_finite];
and a set of actions using:
actInfo = rlFiniteSetSpec([1 2 3 4 5]);
I also created a network called 'actnet' with 4 inputs and 1 output:
But when I try to create an actor using the observations and actions, I am getting an error:
actor = rlDiscreteCategoricalActor(actnet,obsInfo,actInfo);

请先登录，再回答此问题。

回答（1 个）

Narvik
Narvik 2023-8-25
Hi,
I understand that you faced an issue when using a combination of discrete('rlFiniteSetSpec') and continuous('rlNumericSpec') observation data specifications. The function 'rlDiscreteCategoricalActor' accepts a combination discrete and continuous observation data specifications. Please find an example in the documentation below :
https://in.mathworks.com/help/reinforcement-learning/ref/rl.function.rldiscretecategoricalactor.html#mw_b11b3d8e-9739-4cd2-8750-f0350de918f7
I advise you to check your neural network and action space and make sure that the input layers match the number of observation channels. Please find some helpful documentation links provided below :
https://in.mathworks.com/help/reinforcement-learning/ref/rl.util.rlnumericspec.html
https://in.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html
https://in.mathworks.com/help/reinforcement-learning/ref/rl.function.rldiscretecategoricalactor.html
Hope this helps!
  1 个评论
Pruthwiraj Santhosh
Pruthwiraj Santhosh 2024-2-15
Hi Narvik,
Thank you for your response!
I believe the problem is with the mismatch in the calculation of the number of network input layers. But I can't seem to figure out how to calculate this for a heterogenous RL Dataspec object.
For example, If I create obsInfo and actInfo using the following commands
obsInfo_numeric = rlNumericSpec([4 1]);
obsInfo_finite = rlFiniteSetSpec([1 1]);
obsInfo = [obsInfo_numeric,obsInfo_finite];
actInfo = rlFiniteSetSpec([1 2 3 4 5]);
and then create a neural network using the following set of code
actnet = [featureInputLayer(x,"Name","obs") % x - number of input layers
fullyConnectedLayer(20,"Name","fc1")
reluLayer("Name","relu1")
fullyConnectedLayer(20,"Name","fc2")
reluLayer("Name","relu2")
fullyConnectedLayer(20,"Name","fc3")
reluLayer("Name","relu3")
fullyConnectedLayer(5,"Name","act")
softmaxLayer("Name","sfact")]
Can you tell me the value of 'x' in the above code for which the rlDiscreteCategoricalActor won't return an error?
actor = rlDiscreteCategoricalActor(actnet,obsInfo,actInfo,'ObservationInputNames',"obs")

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Agents 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by