Hi Mazbahur,
I understand that you want a single episode to run till each robot either collides with an object or reach the destination(terminating conditions). However currently your episode is being terminated for every robot whenever one of your robot meets the terminating condition.
In a decentralized multi-agent training scenario, where each robot has its own episode termination criteria, it is indeed important to synchronize the termination of episodes to avoid the inefficiency you described. One way to address this issue is by introducing a synchronization mechanism among the robots. Here's a possible approach:
1. Maintain a shared variable or flag that keeps track of whether any robot has reached its episode termination criteria. This flag can be initially set to `False`.
2. Whenever a robot reaches its termination criteria, it checks the shared flag. If it is still `False`, the robot sets it to `True` and waits for a signal from other robots.
3. Meanwhile, other robots continue their execution until they reach their own termination criteria. Once a robot reaches its termination criteria, it checks the shared flag. If it is `True`, the robot proceeds to the next step. Otherwise, it waits until the shared flag becomes `True`.
4. Once all robots have reached their termination criteria and are waiting, a synchronization signal is sent to all robots to indicate that they can start a new episode together. This signal can be broadcasted to all robots simultaneously.
5. Upon receiving the synchronization signal, each robot resets its environment and starts a new episode from the assigned initial positions.
By implementing this synchronization mechanism, you can ensure that all robots wait for each other before starting a new episode. This way, the training of each robot will not be interrupted by the termination of other robots' episodes.
You can create custom Flags using "MATLAB Function Block" in Simulink. Here is a basic example how to use the "Function Block" Implement MATLAB Functions in Simulink with MATLAB Function Blocks - MATLAB & Simulink - MathWorks India.
I hope this helps.
