Using Reinforcement Learning algorithm to optimize parameter(s) of a controller
21 次查看(过去 30 天)
显示 更早的评论
Hello,
First of all, I'm relatively new to reinforcement learning. I have a project in which I need to utilize RL to fine tune one, if not many parameters of an already well built controller, in my case it's a discrete controller. I have came across many papers describing the use of RL specifically for control but not many on optimization which I have trouble in understanding the concept. Perhaps, can someone shed some knowledge on the use of RL for parameters fine tuning, like how is it different than the RL controller concept and how is it going to run in parallel with the controller. I'm more than happy if you can share with me the references, if any. Thanks!
0 个评论
采纳的回答
Emmanouil Tzorakoleftherakis
2020-7-16
Hi Hazwan,
The main difference between using RL for control vs parameter tuning is that in the first case the policy will directly output, e.g. torque, or whatever is your control input. In the latter case, the output of the policy would be parameter values, e.g., if you are trying to tune a PID, the policy would output 3 numbers, Kp, Ki and Kd. Obviously the observations/inputs to the policy as well as the reward would probably need to be different too.
To your question on how the latter could run in parallel with the controller, I can see two scenarios:
1) Using RL for finding static gains. In this case you train, you get the constant parameter values the RL policy finds, and then you discard the policy and adjust your controller gains with these numbers
2) Using RL for finding dynamic/observation-based parameters. This would be in some sense similar to gain scheduling and for this case you would run the policy in parallel with the controller. The idea would be the same(i.e. the policy would output parameter values) but it would do so all the time, thus updating the controller parameters dynamically based on observations.
Hope that helps.
8 个评论
Emmanouil Tzorakoleftherakis
2020-7-22
I would say you need the error yes if you are planning on tracking a collection of step responses. Otherwise, you would be basically overfitting to a single reference value if that makes sense.
You don't need a separate function for the reward, it can be incorporated in the step function as shown in the link you mentioned.
更多回答(1 个)
Mehrdad Moradi
2021-7-26
编辑:Mehrdad Moradi
2021-7-26
But how configure RL to find a static gains while action signal a a time series of different values? Is there any guideline about it?
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Environments 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!