5G Handover with Reinforcement Learning, mismatch of input channels and observations in reinforcement learning representation

Question

Lee Xing Wei 2023-7-12

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1994758-5g-handover-with-reinforcement-learning-mismatch-of-input-channels-and-observations-in-reinforcemen

评论： Emmanouil Tzorakoleftherakis 2023-7-19

Hello,

This is actually my final year project, i have 0 coding knowledge on RL before this.

I am trying to create a custom RL environment in MATLAB. In this environment, I have defined my observation space as rlNumericSpec([numUE*2 1]) because I have numUE user equipment, and each has 2 coordinates (x, y). And my actions are do or not do the handover.

This is what I get when I run the code:

Error using rl.representation.rlAbstractRepresentation/validateModelInputDimension
Model input sizes must match the dimensions specified in the corresponding
observation and action info specifications.
Error in rl.representation.rlQValueRepresentation (line 47)
            validateModelInputDimension(this)
Error in rlQValueRepresentation (line 130)
Rep = rl.representation.rlQValueRepresentation(Model, ...
Error in train2test (line 53)
critic = rlQValueRepresentation(criticNetwork,env.getObservationInfo(),env.getActionInfo(),'Observation',{'state'},'Action',{'action'},criticOpts);

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Lee Xing Wei 2023-7-12

I have tried to chage my observation many times, still get the same error. Actually i want to get the UEpositions, UEvelocity and UEBSconnections as my observations

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Emmanouil Tzorakoleftherakis 2023-7-17

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1994758-5g-handover-with-reinforcement-learning-mismatch-of-input-channels-and-observations-in-reinforcemen#answer_1274293

I suspect you did not set up your critic network properly. If you share that code snippet we can take a closer look. An alternative would be to use the default agent feature and let the software create a critic for you automatically based on provided observation and action spaces. Here is an example that assumes you want to create a DQN agent:

https://www.mathworks.com/help/reinforcement-learning/ref/rl.agent.rldqnagent.html#mw_4b8dc3d0-2667-44d5-ba22-aeae022bda64

3 个评论
显示 1更早的评论隐藏 1更早的评论

Lee Xing Wei 2023-7-19

Sir, please help me to check

Emmanouil Tzorakoleftherakis 2023-7-19

在 MATLAB Online 中打开

As I mentioned, you can create the agent using the default agent feature like this:

agent = rlDQNAgent(obsInfo,actInfo)

If you run this line, it won't create any errors.You can then check what the neural network look slike by doing:

critic = getCritic(agent);
criticNet = getModel(critic);
plot(criticNet);

That said, take another look at how you defined your action space. If your output is only 0 and 1, your action space is not defined correctly (it's currently a cell array of [0 1]).

请先登录，再进行评论。