How Unreal Engine Simulation for Robots Works

Robotics System Toolbox™ provides a co-simulation framework that you can use to model robotics algorithms in Simulink^® and visualize their performance in a virtual simulation environment. This environment uses the Unreal Engine^® by Epic Games^®.

Note

Simulating models in the 3D visualization environment requires Simulink 3D Animation™.

Understanding how this simulation environment works can help you troubleshoot issues and customize your models.

Communication with 3D Simulation Environment

When you use Robotics System Toolbox to run your algorithms, Simulink co-simulates the algorithms in the visualization engine.

In the Simulink environment, Robotics System Toolbox:

Configures the visualization environment, specifically the ray tracing, scene capture from cameras, and initial object positions
Determines the next position of the objects by using the simulation environment feedback

The diagram summarizes the communication between Simulink and the visualization engine.

During 3D simulation, Simulink determine positions of objects and configure 3D environments. Simulink then sends translation, rotation, and scale information to Visualization engine. Visualization engine position objects in 3D environment and query 3D environment. Visualization engine then sends scene information back to Simulink

Block Execution Order

During simulation, the Unreal Engine simulation blocks follow a specific execution order:

The Simulation 3D Robot and Simulation 3D Physics Dump Truck blocks initialize the vehicles and send their Translation, and Rotation signal data to the Simulation 3D Scene Configuration block.
The Simulation 3D Scene Configuration block receives the vehicle data and sends it to the sensor blocks.
The sensor blocks receive the vehicle data and use it to accurately locate and visualize the vehicles.

The Priority property of the blocks controls this execution order. To access this property for any block, right-click the block, select Properties, and click the General tab. By default, Simulation 3D Robot and Simulation 3D Physics Dump Truck blocks have a priority of -1, Simulation 3D Scene Configuration blocks have a priority of 0, and sensor blocks have a priority of 1.

The diagram shows this execution order.

Execution order of the Unreal Engine Simulation blocks

If your sensors are not detecting vehicles in the scene, it is possible that the Unreal Engine simulation blocks are executing out of order. Try updating the execution order and simulating again. For more details on execution order, see Control and Display Execution Order (Simulink).

Also be sure that all 3D simulation blocks are located in the same subsystem. Even if the blocks have the correct Priority settings, if they are located in different subsystems, they still might execute out of order.