Different Sample time for RL environment

Question

Syed Adil Ahmed 2024-6-11

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2127656-different-sample-time-for-rl-environment

评论： Syed Adil Ahmed 2024-6-13

Hello Everyone,

I am trying to build a RL agent using DQN currently. My environment model is composed of Nonlinear equations which have faster dynamics around 1ms. I was wondering if it possible to have my RL agent to work at 10ms and let the environment run at 1ms. This will obviously mean that for 10 timesteps the environment will be feeding at a constant control action from the RL agent.

I know I can code this up if I create the agent and environment myself, but currently I'm taking advantage of MATLAB's Reinforcement Learning Toolbox in MATLAB editor and I would save a lot of time if this is possible in the toolbox itself.

Thanks.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Kartik Saxena 2024-6-13

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2127656-different-sample-time-for-rl-environment#answer_1471376

在 MATLAB Online 中打开

Hi,

When creating a custom environment in MATLAB for use with the Reinforcement Learning Toolbox, you can define a 'step' function that advances the environment's state based on the agent's action. To simulate the environment at a faster rate than the agent operates, modify the 'step' function to perform multiple updates (10 updates of 1ms each) for each call to the 'step' function.

Refer to the following code snippet:

    for i = 1:10
        % Update the environment state based on the action
        % Assume updateEnvironment is a function that updates the environment state
        % for a 1ms timestep given the current state and action
        loggedSignals = updateEnvironment(loggedSignals, action);
    end