From your figure, the discounted reward value is very large. try to rescale it to a certain value [-10, 10] in the environment. For example, r(t) = 10 * Microgrid operational cost (t) / MaxCost , where MaxCost is the maximum possible cost per time step.
Another point is you can use another agent.
I hope these suggestions can solve your concerns.