Splitting the input layer of deep neural network (used for the actor of a DDPG agent)

Question

Nima Mathwork 2020-6-25

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/554611-splitting-the-input-layer-of-deep-neural-network-used-for-the-actor-of-a-ddpg-agent

评论： Heesu Kim 2021-1-21

Hello everyone

I am using the DDPG agent to control my robot. I want to design a neural network with architecture similar to the figure below for my actor. Ideally, I want to deploy an imageInputLayer with size [17 1 1] as inputs and then simply split these inputs into two branches, which each one connected only to nine elements of inputs(one element is shared) and ends at a different output neuron. Finally, these two neurons should be concatenated. I appreciate it if someone illustrates how I can do this.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Anh Tran 2020-9-18

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/554611-splitting-the-input-layer-of-deep-neural-network-used-for-the-actor-of-a-ddpg-agent#answer_496936

在 MATLAB Online 中打开

You can define 2 observation specifications on the environment. Thus, the agent will receive splitted input to begin with. Moreover, since your observation are vector-based, you can try featureInputLayer (R2020b) instead of imageInputLayer.

obsInfo1 = rlNumericSpec([9,1]);
obsInfo2 = rlNumericSpec([9,1]);
obsInfo = [obsInfo1 obsInfo2];

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Heesu Kim 2021-1-21

在 MATLAB Online 中打开

Hi.

Is there any other things that must be modified following the obs separation?

I am trying actor-critic model with separate observation input (exactly the same as the question), and modified actor and critic object as following.

(before)

actor = rlStochasticActorRepresentation(actorNetwork,obsInfo,actInfo,...
    'Observation',{'state'},actorOpts);

(after)

actor = rlStochasticActorRepresentation(actorNetwork,obsInfo,actInfo,...
    'Observation',{'state1, state2'},actorOpts);

However, I'm getting an error like

Caused by:
    Error using
    rl.representation.rlAbstractRepresentation/validateInputData
    (line 507)
    Input data must be a cell array of
    compatible dimensions with observation
    and action info specifications.

I was not able to find where should I change.

Is there something else to be modified following the obs separation?

请先登录，再进行评论。

Splitting the input layer of deep neural network (used for the actor of a DDPG agent)

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Splitting the input layer of deep neural network (used for the actor of a DDPG agent)

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论