ExperienceBufferLength in Reinforcement Learning Toolbox
显示 更早的评论
Hello, everyone,
I found a problem with the 'ExperienceBufferLength' property in 'rlDDPGAgentOptions' when specifying options for rl agents.
In this example, every episode has 600 (60/0.1) steps. Does the agent start to train when the experience buffer is filled up with the experiences (S,A,R,S'). If so, it would take at least 1667 (1000000/600 ) episodes before the agent starts to improve.
So I want to know how to determine this value.
采纳的回答
Ari Biswas
2021-11-17
0 个投票
The agent will train until at least one minibatch can be sampled from the buffer. If your mini batch size is 64, then the first learn step will occur after the buffer has stored 64 experiences. The experience buffer is circular, i.e., it removes older experiences when full. The size of the buffer is hence important. You may lose important experiences if the buffer size is too small.
4 个评论
Soheil Khoshboo
2022-8-9
Dear Ari,
hope your doing well. I have seen one of your videos on mathworks website under the title of "Reinforcement Learning for Ball Balancing Using a Robot Manipulator". It was a really good video, thanks for sharing your knowledge. Now I was checking for the model file on the website but unfortunately I could not find it, I appreciate if you send me files for that tutorial so that I can work on it and understand it better
regards
Ari Biswas
2022-8-9
https://www.mathworks.com/help/reinforcement-learning/ug/train-sac-agent-for-ball-balance-control.html
Arman Ali
2022-9-27
How about if we want to fill our buffer first and then start taking minibatches?? how to implement this in matlab?
Francisco Serra
2024-5-2
For that you can set:
agent.AgentOptions.NumWarmStartSteps=experience_buffer_length
As default, this is set to the minibatch size, but changing to the experience buffer size will force the algorithm to wait until the buffer is full.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
