Time-varying policy function

14 次查看（过去 30 天）
Matheus Silva
Matheus Silva 2023-5-24
评论： Emmanouil Tzorakoleftherakis 2023-5-30
Hi,
I am wondering if it is possible to have time-varying (non-stationary) policy functions in the reinforcement learning toolbox.
For example, say my episode lasts three periods (t=1,2,3), then I would have the set where is some neural network structure indexed by a general vector of parameters ϑ, which will ultimately depend on the time period.
Is that possible to do with the toolbox?
Thank you so much!

请先登录，再回答此问题。

回答（1 个）

Emmanouil Tzorakoleftherakis
Why don't you just train 3 separate policies and pick and choose as needed?
  4 个评论
显示 2更早的评论
Matheus Silva
Matheus Silva 2023-5-28
编辑：Matheus Silva 2023-5-28
My problem is that my periods can be related in some arbitrary way. For example, I am thinking of a model where the state can vary according to
Where is a stochastic term and is some transition function. However, I may want to allow some relation between the stochastic terms in periods 1 and 3. Solving the problem period by period would eliminate that dependence, no?
Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2023-5-30
Honestly, I think your best bet would be to use the same policy throughout, but maybe use an input signal to the neural net to indicate which period you are in based on your state.
Another option, which is similar to what I mentioned earlier, is to train 3 different policies. To work around the period dependencies, you can place the RL policy block inside a triggered subsystem and only enable the subsystem for training when the system is in the appropriate period. Do that for each policy and then you can switch between the 3 as needed. See here

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by