PPO convergence guarantee in RL toolbox
4 次查看(过去 30 天)
显示 更早的评论
Hi,
I am testing my environment using the PPO algorithm in RL toolbox, I recently viewed this paper: https://arxiv.org/abs/2012.01399 which listed some assumptions on the convergence guranteen of PPO, some of them are for the environment itself (like the transition kernel...) and some are for the functions and parameters of the algorithm (like the learning rate alpha, the update function h...)
I am not sure if the PPO algorithm in the RL toolbox satisfies the assumptions of the convergence for the functions and parameters of the algorithm, because I did not find any direct mentioning of convergence in the official mathwork website, so I wonder how the algorithm is designed such that convergence is being considered.
Do I need to look into the train() function to see how those parameters and functions are designed?
Thank you
0 个评论
回答(1 个)
Karan Singh
2024-6-17
Hi Haochen,
The Proximal Policy Optimization algorithm in MATLAB's Reinforcement Learning Toolbox is based on the foundational principles from the original PPO papers by Schulman et al. (2017), as referenced in the documentation (https://www.mathworks.com/help/reinforcement-learning/ug/proximal-policy-optimization-agents.html).
It's crafted to adhere to the core assumptions necessary for the algorithm's convergence under certain conditions. However, the success of PPO, like many RL algorithms, hinges on several factors, including hyperparameter settings, the environment's complexity, and the implementation details.
Regarding the source code, accessing the detailed internals of the implementation might not be possible.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Statistics and Machine Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!