The agent can learn the policy through the external action port in the RL Agent so that the agent mimics the output of the reference signal

Question

凡 2023-9-11

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2019326-the-agent-can-learn-the-policy-through-the-external-action-port-in-the-rl-agent-so-that-the-agent-mi

评论：凡 2024-2-26

I created a DDPG agent that I wanted to learn from the output of an existing controller before training it later. So, I input the reference signal through the external action port, and set the use external action to 1 for training, when training, the output of the agent is the reference signal, but after the training. When I set the use external action to 0 for verification, the output of the agent is not the same as the reference signal, and the difference is a bit big. Does the external action port work with my idea? What should I do to realize my idea?

The figure below shows that when the external action is set to 0, the output of the trained agent is a red curve, and the reference signal is a green curve

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Emmanouil Tzorakoleftherakis 2023-9-25

It seems the agent started learning how to imitate the existing controller but needs more time. What does the Episode Manager look like? What is your reward signal?

2 个评论
显示无隐藏无

凡 2024-2-26

Meaning the idea is feasible, but may it need more training Episodes?

凡 2024-2-26

This is the Episode Manager，My bonus signal is: -4*u^2-du/dt，u is an observational measurement，My control goal is to make u 0. My project is to replace the PID controller with an agent，In PID control，u is the input quantity，So I want the agent to mimic the output of the PID at the beginning