Hessian in mlecov compared to hessian from fmincon

1 次查看（过去 30 天）
zym
zym 2022-8-27
评论： Bruno Luong 2022-8-27
Hi, I am doing custom MLE estimation using fmincon. I want to compute asymptotic covariance matrix of MLE. To do so I could use mlecov or Hessian from fmincon. I was wondering if the hessian is estimated in mlecov in the same way as in fmincon? If not, what are possible advantages of mlecov?
Since I am running a custom problem, mlecov gives me an issue about positive definiteness. I am thinking to opt for the hessian as coming from the fmincon. However, the documentation suggest that the hessian output from fmincon is not precise https://nl.mathworks.com/help/optim/ug/hessian.html . What are other possible options? I identify three:
  1. Use the maximised value of parameters from fmincon and plug it in fminunc to obtain the hessian, which seems to be correctly estimated there. Update: I still find that the Hessian is not-positive definite
  2. Correct the hessian in mlecov to be positive definite (though I am not sure if it is a reliable procedure) Update: In the comment below, Matt J suggests a great function to that.
  3. Drop using fmincon. Since my constraints are only non-negativity of variance, transform parameters and just use fminunc. Update: Hessian from fminunc still comes out to be non positive definite. However, mlecov does the magic. Once I plugin maxima from fmincon in the yet another custom function where parameters are transformed, such the problem is unconstrained, I find that mlecov is able to give me a positive-definite Hessian.
Did any of you happenned to have the same issue?
Thanks for your help. Regards!
  2 个评论
Matt J
Matt J 2022-8-27
编辑：Matt J 2022-8-27
You could also just compute the Hessian yourself, analytically.
zym
zym 2022-8-27
编辑：zym 2022-8-27
Thanks, it is a good point. But my problem is too complicated for the analytical solutions. Includes Kalman filter etc...

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Matt J
Matt J 2022-8-27
编辑：Matt J 2022-8-27
I was wondering if the hessian is estimated in mlecov in the same way as in fmincon?
Probably, but even if not, the Hessian inversion will be sensitive to errors from finite difference computation.
You could try nearestSPD to find the nearest positive semi-definite matrix to whatever fmincon gives you,
https://www.mathworks.com/matlabcentral/fileexchange/42885-nearestspd?s_tid=ta_fx_results
Be mindful of a a few things, though,
(1) A positive semi-definite matrix is still not positive definite.
(2) The theory of approximating MLE covariance with Hessians is only valid when no constraints are active at the solution.
  4 个评论
显示 2更早的评论
zym
zym 2022-8-27
Thanks for this. However, trust region algorithm needs analytical gradient, which I do not have.
To obtain asymptotic covariance matric, mlecov computes Hessian using final differences. Inside the function, they write:
% Compute pure and mixed second order central differences of the
% log-likelihood function. H is always a square matrix, regardless of
% the shape of params. nH is the negative hessian, because loglikefun
% returns a negative log-likelihood.
Bruno Luong
Bruno Luong 2022-8-27
If that is the case, I would trust the Hessian returned by mlecov much more.
The Hessian used and returned by optimlization algorithms are designed for ... optimization, meaning the main purpose is to find a correct descend direction and not too costly. So I would not use it to make any further fine analysis.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Quadratic Programming and Cone Programming 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by