Hi Nicolas,
Your approach to analyze the learning curves is indeed a good approach to identify possible issues in the training of a reinforcement learning model.
The conclusion of the question asked might also depend on other factors but one can interpret a few things by analyzing these curves.
It seems like training is stuck between lower episode reward and might converge to a suboptimal policy. It seems like you have to increase the complexity of the model by increasing the learnables.
The selection of a learning rate is an iterative process and you can experiment with different values and analyze the results. It looks like you might have a slightly higher learning rate as the learning curve is oscillating between different values.
You might want to consider the structure of the reward function as it might not be communicating the desired outcome to the agent resulting in irregular improvement in learning curve.
You might want to experiment with different values for hyperparameters, learning rates and complexity of model and analyze their impact on the learning curve.
I hope it helps!
