How to connect simulink real-time(or step-by-step) control with python for RL

Question

원혁 최 2022-6-22

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1745560-how-to-connect-simulink-real-time-or-step-by-step-control-with-python-for-rl

回答： Pratik 2024-1-3

采纳的回答： Pratik

I'm trying to connect simulink model with python to control the model with Reinforcement learning

For this, I need to simulate a few timesteps (for example, 0.01hr)

then python reads current states and sets the control value by using policy / also learning and optimizing policy

I am currently using the pause method with matlab engine API:

By using clock and assertion block, if simulation time reaches that specific interval(0.01hr), simulation pauses

python checks the simulation status constantly, and if simulation is paused, it reads state values, set control, learns

and then resume simulation.

This works quite well, but repeatedly pausing simulation spends too much time.

Doing 5hr simulation 6000 times takes 3 whole days.

Is there any way better/faster?

I tried TCP-IP connecting but I don't have lincense for corresponding toolbox and I think control action cannot be applied for every fixed timestep for its real-timedness.

Thanks in advance.

2 个评论
显示无隐藏无

Sijin 2022-11-30

Hi,

Did you find a solution for this? I am having a similar usecase, and currently using the pause to update the control action from Python. As you mentioned, it takes a lot of time for a meaningful number of training steps.

Please share if you have found some better solution.

Best Regards,

Sijin

Enzo Yacometti 2022-12-13

Hello, if anyone has a solution to this I would be very interested. I'm currently also pausing and resuming constantly the simulation. Thank you.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Pratik 2024-1-3

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1745560-how-to-connect-simulink-real-time-or-step-by-step-control-with-python-for-rl#answer_1382301

Hi 원혁 최,

As per my understanding, you want to connect a Simulink model with python to control the model with Reinforcement learning. Using the MATLAB engine API and pausing the Simulink model makes the whole process slow.

To run Python code in a Simulink model, a MATLAB function block or a MATLAB System block can be used. To call Python modules in MATLAB, there should be a supported version of the reference implementation (CPython) installed on the system.

Please refer to the following documentation for an example usage of a MATLAB function block and a MATLAB System block:

www.mathworks.com/help/simulink/ug/python_simulink_example.html#responsive_offcanvas

Please refer to this question on MATLAB answers for more information:

www.mathworks.com/matlabcentral/answers/359575-how-to-run-a-python-code-on-simulink

I Hope this helps.