Check Gradient (Jacobian) of objective function -- what is the meaning of absolute difference (1e-6) if component of gradient is less than 1 ?

4 次查看（过去 30 天）
SA-W
SA-W 2023-1-18
编辑： Torsten 2023-1-18
I use lsqnonlin to solve my data-fitting problem and provide the Jacobian, which I verify using CheckGradients option.
As stated here, if a component of the Jacobian is less than 1, gradient check is successful if the absolute difference between the user-shipped Jacobian and Matlabs finite-difference approximation of that component is less than 1e-6.
Example: Jacobian_USER(1,1)=2e-7, Jacobian_MATLAB(1,1)=1e-7.
Here, Jacobian_USER(1,1) is clearly wrong, but Jacobian check would be successful. Is that true?
  11 个评论
显示 9更早的评论
SA-W
SA-W 2023-1-18
"You shouldn't assume something is wrong if the result of one computation gives 2e-7, the result of another computation gives 1e-7."
Why should I not be alarmed in that case?
If I know my values are so small (around 1e-7) per nature, a difference between 1e-7 and 2e-7 is significant.
Torsten
Torsten 2023-1-18
编辑：Torsten 2023-1-18
Why should I not be alarmed in that case?
If I know my values are so small (around 1e-7) per nature, a difference between 1e-7 and 2e-7 is significant.
The values you investigate are numerical derivatives (usually of complex functions). Already when choosing the h in approximating the derivative as (f(x+h)-f(x))/h can cause such differences.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Matt J
Matt J 2023-1-18
编辑：Matt J 2023-1-18
If I know my values are so small (around 1e-7) per nature, a difference between 1e-7 and 2e-7 is significant.
Not necessarily. The element-wise relative error is large, but remember lsqnonlin uses the Jacobian J in matrix products of the form J*x and J.'*y to compute its updates. If the average value of abs(J(m,n)) is 1000 and the average values of abs(x(n)) or abs(y(m)) is 10, then an absolute error of 1e-7 in any entry J(m,n) would likely contribute negligibly to J*x or J.'*y.
Regardless, if you don't have full confidence in DerivativeCheck, you can implement your own finite difference computation, using for example,
https://www.mathworks.com/matlabcentral/fileexchange/13490-adaptive-robust-numerical-differentiation
and then do the comparison according to a criterion you prefer.

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Surrogate Optimization 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by