The gradient is the gradient of the square of the error function
error = (knowntarget - variableoutput)
with respect to the unknown weights and biases.
Typically, the training objective is to optimize te choice of weights nd biases by minimizing the sum of the squared errors by using the method of steepest descent.
However, to slow the speed of the descent so that the search value does not fly back and forth across the minimum without stopping sufficiently near it, a momentum term is added. For details search Google using
GRADIENT DESCENT WITH MOMENTUM
Hope this helps
Thank you for formally accepting my answer
Greg
