Hello Muhammad,
I understand that you are facing an issue where the episode “Q0” reward is becoming positive though it was designed to achieve negative reward, to address this issue, I suggest considering the following solutions:
- Scale between “Q0” and episode reward: It is possible that there is a significant difference in scale between the “Q0” estimate and the actual episode reward. This disparity may lead to unexpected results and impact the training process. To investigate this, you can try unchecking the "Show Episode Q0" option to see if it affects the episode reward values.
- Another possibility is that there might be an issue with the implementation of the DDPG algorithm itself. The algorithm should be able to handle both positive and negative rewards. It is important to ensure that you are using the return, which is the sum of the rewards for a specific state-action pair from that point until the end of the trajectory.
- Simplify the critic network: It might be helpful to simplify the critic network to ensure that it outputs values on a similar scale as the episode reward. This can help align the “Q0” estimates with the actual rewards, providing more accurate feedback for the agent's learning process.
Further you can have a refer to the MathWorks Documentation:
I hope this helps!
