Issue with large memory required for non-linear optimizer

Question

Yannis Stamatiou 2023-4-18

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1948908-issue-with-large-memory-required-for-non-linear-optimizer

评论： Yannis Stamatiou 2023-4-20

Dear Matlab community hi.

I tried to run the following optimization problem for a 2-dimensional optimization variable of size 150x150. For some reason, the system creates somehow in the optimization process (I guess) some matrix of size (150^2)x(150^2). I tried to solve the issue for several days now (with the different options shown in comments) but I cannot understand why MATLAB creates such a huge matrix in the solution process. Is there, perhaps, some other nonlinear optimizer in MATLAB that does not require such huge matrices? Any help on this issue would be very helpful.

With best wishes,

Yannis

a = 4;
b = 2.1;
c = 4;
x = optimvar('x',150,150);
prob = optimproblem;
prob.Objective = parameterfun(x,a,b,c);
%opts=optimoptions('fmincon','Algorithm','interior-point','SpecifyObjectiveGradient',true,'HessianFcn','objective');
%opts=optimoptions('quadprog','Algorithm','trust-region-reflective','Display','off');
opts = optimoptions('fminunc','Algorithm','trust-region');
opts.HessianApproximation = 'lbfgs';
opts.SpecifyObjectiveGradient = false;
x0.x = 0.5 * ones([150,150]);
%[sol,qfval,qexitflag,qoutput] = solve(prob,x0,'options',opts);
[sol,fval] = solve(prob,x0)

3 个评论
显示 1更早的评论隐藏 1更早的评论

Yannis Stamatiou 2023-4-20

移动：John D'Errico 2023-4-20

Hi,

thank you all for your replies!

With best wishes,

Yannis

Torsten 2023-4-20

移动：John D'Errico 2023-4-20

And what did you decide to do ?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Alan Weiss 2023-4-19

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1948908-issue-with-large-memory-required-for-non-linear-optimizer#answer_1218938

You have 150^2 optimization variables. I do not see your parameterfun function, but if it is not a supported function for automatic differentiation, then fminunc cannot use the 'trust-region' algorithm because that algorithm requires a gradient function. The LBFGS Hessian approximation is not supported in the 'quasi-newton' algorithm. Sorry.

Alan Weiss

MATLAB mathematical toolbox documentation

6 个评论
显示 4更早的评论隐藏 4更早的评论

Bruno Luong 2023-4-19

在 MATLAB Online 中打开

@Alan Weiss "The LBFGS Hessian approximation is not supported in the 'quasi-newton' algorithm. Sorry."

Tiy probably want to say "Trust-region" algorithm, since LBFGS and quasi-newton are often recommended and used for large sale problem.

This code is modified version of example from the help page

N = 1e5;
x0 = -2*ones(N,1);
x0(2:2:N) = 2;
options = optimoptions("fminunc",Algorithm="quasi-newton", HessianApproximation="lbfgs",...
    SpecifyObjectiveGradient=true);
[x,fval] = fminunc(@multirosenbrock,x0,options);
Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
function [f,g] = multirosenbrock(x)
% Get the problem size
n = length(x);  
if n == 0, error('Input vector, x, is empty.'); end
if mod(n,2) ~= 0
   error('Input vector, x ,must have an even number of components.');
end
% Evaluate the vector function
odds  = 1:2:n;
evens = 2:2:n;
F = zeros(n,1);
F(odds,1)  = 1-x(odds);
F(evens,1) = 10.*(x(evens)-x(odds).^2); 
f = sum(F.^2);
if nargout >= 2 % Calculate gradient
    g = zeros(n,1);
    g(evens) = 200*(x(evens)-x(odds).^2);
    g(odds) = -2*(1 - x(odds)) - 400*(x(evens)-x(odds).^2).*x(odds);
end
end

Bruno Luong 2023-4-20

@Yannis Stamatiou " I cannot figure out exactly why"

The lbfgs formula approximate the inverse of the Hessian by low-rank approximation and does not require to store the full Hessian or its inverse.

That's why the memory requirement is reduced and it is suitable for lare-scale problem.

Yannis Stamatiou 2023-4-20

Hi,

I now understand why it worked. In fact it is not very easy to see what is the best approach for large optimization problems, I was a bit lost in the documentation with the several optimization options, algorithms and settings. However, all the replies to my post, and yours in particular with respect to why the method worked, were very helpful, thanks!

Yannis

请先登录，再进行评论。