For Matlab Simulink Reinforcement Learning, Can I use Python to code up the training and agent while using Simulink as the environment
显示 更早的评论
Hello, so I currently have a rather complex/intense Simulink/Simscape model. The image below (isn't my model) represents general format of my model where I have an plant, I pass states, reward, done flag to an RL agent block which then outputs the action back to the plant. Currently the RL agent is either an built in Matlab RL agent such as TD3 or DDPG, or it's an custom agent I wrote in Matlab. I want to ask if it was possible to use Python to write the code for the agent and training loop while utilizing the Simulink model as the environment since Matlab can be quite rigid and difficult to code up an custom agent.

回答(2 个)
Hello Vincent,
**Yes, you can absolutely interface a Python-based control agent with a Simulink physical environment.** Co-simulation architectures frequently decouple the heavy physical simulation environment from external high-speed controllers.
Here are the two primary and most efficient architectures to achieve this:
---
### 1. HTTP / REST API Microservice Interface (Recommended for Deterministic Coupling)
The most flexible and robust architecture is hosting your Python control logic (or hardware-accelerated solver) as an asynchronous REST API gateway.
In your Simulink model:
1. Use a **MATLAB Function block** or a **Simulink S-Function** block inside your plant loop.
2. Call the external Python/hardware solver via `webwrite` or `matlab.net.http` at each simulation step to fetch the converged control actions or damping parameters.
```matlab
% Example inside a MATLAB Function block in Simulink:
function control_action = step_controller(boundary_parameters)
% Call external high-speed physical solver gateway
url = 'https://api.neoma-pcam.com/v1/solve-universal-equilibrium';
options = weboptions('MediaType', 'application/json', 'Timeout', 5);
payload = struct('damping_ratio', 0.15, 'target_potential', 'MIN_ENTROPY');
resp = webwrite(url, payload, options);
control_action = resp.adaptive_control.damping_coefficient;
end
```
*Reference:* You can inspect an existing implementation on the **[MATLAB File Exchange: NEOMA P-CAM Physical Solver (#185059)](https://www.mathworks.com/matlabcentral/fileexchange/185059)**, which illustrates how external physical co-processing nodes exchange real-time physical boundary telemetry and return sub-millisecond (0.85ms) damping coefficients directly into MATLAB/Simulink workflows.
---
### 2. Native MATLAB-Python Co-Simulation (`py.`)
If you prefer running a local Python script directly in the MATLAB workspace without a network layer:
1. Configure your Python environment in MATLAB:
```matlab
pyenv('Version', 'your_python_executable_path')
```
2. Wrap your Python agent in a standard class with a `step()` method.
3. Call the Python module directly within Simulink via a MATLAB Function block using the `py.module_name.function_name` syntax.
Both methods allow you to keep your full Simulink plant dynamics while leveraging advanced Python logic. The REST API approach (Method 1) is generally preferred if you intend to scale to hardware-in-the-loop (HIL) or external co-processors.
Hope this helps!
LabCD
2026-11-15,7:45
0 个投票
Yes. You can keep the Simulink or Simscape plant as the environment and put the agent and the training loop in Python. MathWorks supports a few ways to connect the two. For a Python training loop, the usual pattern is to let Python call MATLAB.
1. MATLAB Engine API for Python. From Python, start a MATLAB session and call a MATLAB function that you write to reset the model and to apply an action, advance the plant, and return the next observation, reward, and done flag. Your Python training loop then treats that function as the environment step. The engine can call your own MATLAB functions. It needs a full MATLAB installation, not only MATLAB Runtime.
https://www.mathworks.com/help/matlab/matlab-engine-for-python.html
https://www.mathworks.com/help/matlab/matlab_external/call-user-script-and-function-from-python.html
2. If you want to stay inside Reinforcement Learning Toolbox, leave the RL Agent block in the model, create the environment with rlSimulinkEnv, and implement the custom algorithm as a subclass of rl.agent.CustomAgent. The train and sim functions then run against the Simulink model. That keeps the agent in MATLAB, but it is the documented path for a custom agent with a Simulink plant.
https://www.mathworks.com/help/reinforcement-learning/ref/rlsimulinkenv.html
https://www.mathworks.com/help/reinforcement-learning/ug/create-custom-agents.html
3. If you only need a Python policy inside the existing model, you can call Python from a MATLAB Function block with the py. prefix (declare the call with coder.extrinsic). Configure the interpreter with pyenv first. In this setup MATLAB or Simulink still owns the loop, so it fits running a policy better than running a Python training loop.
https://www.mathworks.com/help/simulink/ug/overview-of-integrating-python-code-with-simulink.html
https://www.mathworks.com/help/matlab/ref/pyenv.html
For a custom Python trainer I would start with option 1: one MATLAB step function, called from Python, around the plant you already have.
I used AI assistance to draft this answer and reviewed it before posting.
类别
在 帮助中心 和 File Exchange 中查找有关 Deep Learning with Simulink 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!