alternatives to gradient-based optimization algorithms like lsqnonlin that handle at least O(10) parameters?

37 次查看（过去 30 天）

SA-W 2023-2-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1918370-alternatives-to-gradient-based-optimization-algorithms-like-lsqnonlin-that-handle-at-least-o-10-par

评论： SA-W 2023-3-6

I am fitting parameters of a PDE to experimental data utilizing the function lsqnonlin from the optimization toolbox.

I observed that I can successfully optimize 10,...,20 parameters but not more. My objective is, however, to optimize 50 to 100 parameters.

Based on my experience with lsqnonlin (and fmincon and friends,...), it seems that this class of optimization algorithms can handle a small number of parameters (10,...,20) well, but are not appropriate anymore if there are many more parameters.

Fast convergence or computation time are not important to me.

I am coming from an engineering background and do not have deep knowledge about other class of optimization algorithms. That said, I would appreciate if someone could give me some keywords or recommendations regarding alternative optimization algorithms, that are designed for handling a larger number of parameters.

Thank you!

18 个评论
显示 16更早的评论隐藏 16更早的评论

SA-W 2023-2-24

How have you assessed whether they are "wrong"? What is the final objective function value, and how does it compare to the objective value of the "right" parameters?

The objective function value is indeed in the order of the objective value corresponding to the right parameters, however, the parameters are clearly wrong which I observed by plotting the function.

The parameters that I am optimizing (the function values) have to represent a convex function in 1d. However, the 20 parameters found by lsqnonlin do not define a convex function, but rather a function which is at certain ranges even decreasing, which, for whatever reasons, is also a minimum of the objective function.

Let me ask two questions:

(i) is there a recommended ratio (just roughly) between number of parameters and equations? Currently, my data vectors have 1500 entries and I optimise 20 parameters.

(ii) I am considering enforcing convexity by having the following constraints on the parameters p(i):

p(i-1)-pf(i)+p(i-1) > 0 for all parameters

Is there a way to retranslate this constraints for lsqnonlin, for instance by adding a penalty term to

f = ||r||^2 + penalty term?

Matt J 2023-2-26

编辑：Matt J 2023-2-26

You could do that, or even have c(i) = (p(i-1)-2p(i)+p(i-1))^200. However, with larger exponents, the gradient of the cost function become small over a wider neighborhood of c=0. Smaller gradients means slower convergence, and also the chance that the OptimalityTolerance would trigger prematurely. Conversely, with smaller exponents (but stil greater than 1), the gradient becomes less smooth. So, there is a trade-off to be reckoned with.

SA-W 2023-3-6

编辑：SA-W 2023-3-6

在 MATLAB Online 中打开

@Matt J

I implemented

r=[weight1*r; weight2*constraint_violation];
f = ||r||^2 

iwith lsqnonlin and figured out that

weight_2 = 1e4 %approximately

is necessary that the ineqaulity constraints are considered at all at intermediate iterations and weight_2 below or above 1e4, respectively, will not include sufficiently the constraint_violation or dominates the residual r.

If I implement the above inequality constraints in fmincon and multiply A*x<=b by 1e4 (this multiplication is necessary for fmincon to pay attention to the constraints), fmincon gives a better solution than lsqnonlin. By better, I mean that the solution is nearly perfect equal to the exact solution. The lsqnonlin solution fulfills the constraints in A, however, the solution is not as accurate.

Anyway, fmincon requires twice or three times as much iterations as lsqnonlin requires to find the solution. That said, I would like to stick lsqnonlin because the evaluation of the objective function is quite expensive in my case.

I read the documentation on the iterior-point algorithm. The way fmincon incorporates the constraints is based on a merit function, which is, of course, different than just expanding the residual vector as I did it.

Admittedly, I do not have enough background in optimization. Can you give me another recommendation (which is closer to the way fmincon incorporates the constraints) to implement my constraints in lsqnonlin?

Thank you!

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Matt J 2023-2-24

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1918370-alternatives-to-gradient-based-optimization-algorithms-like-lsqnonlin-that-handle-at-least-o-10-par#answer_1179270

编辑：Matt J 2023-2-24

it seems that this class of optimization algorithms can handle a small number of parameters (10,...,20) well, but are not appropriate anymore if there are many more parameters.

I am currently using lsqnonlin in optimization problems with 11796480 variables. It converges fine, though in my case the line search operations really do make it much slower than I would like. In any case, 20 variables is definitely not the upper limit.

I am fitting parameters of a PDE to experimental data utilizing the function lsqnonlin from the optimization toolbox.

Make sure you are following the guidelines here,

https://www.mathworks.com/help/optim/ug/optimizing-a-simulation-or-ordinary-differential-equation.html