Hi Yunyi,
When training reinforcement learning agents using the reinforcement learning toolbox in MATLAB, it's important to manage the memory usage to avoid filling up the C drive. Here are a few methods you can try to mitigate the increasing disk space usage during training:
Change the temporary directory: MATLAB uses a temporary directory for storing intermediate data during training. By default, this directory is set to the system's temporary folder, which is often located on the C drive. You can change the temporary directory to a different location, such as a different drive with more available space. Use the tempdir function in MATLAB to set a different temporary directory.
newTempDir = 'D:\Temp'; % Replace with the desired temporary directory
matlab.internal.getAndSetTemporaryDirectory(newTempDir);
- Limit the size of the experience replay buffer: The experience replay buffer can consume a significant amount of memory during training. You can reduce its size by adjusting the BufferSize parameter when creating the rlDQNAgent or rlDDPGAgent object. A smaller buffer size will require less memory but may affect learning performance.
- Reduce the training batch size: The batch size determines how many experiences are processed in each training iteration. A smaller batch size will reduce the memory footprint but may slow down training. You can adjust the MiniBatchSize parameter when creating the agent to decrease the batch size.
- Lower the resolution of observations: If the observations in your environment have high resolution, consider reducing their resolution. This can be achieved by downsampling or compressing the observations before feeding them into the agent. Lower resolution observations require less memory.
- Restart MATLAB periodically: As you mentioned, restarting MATLAB releases the memory consumed during training. If it's feasible for your workflow, you can periodically restart MATLAB after a certain number of training iterations or episodes to reclaim the memory. You can use the exit function to gracefully terminate MATLAB.
These methods should help mitigate the increasing disk space usage during training. It's important to find the right balance between memory usage and training performance for your specific application.
