Loading and saving variables when using parallel toolbox
1 次查看(过去 30 天)
显示 更早的评论
I am solving a systme of equations using fsolve, where in order to compute those equations I need to solve a recursive problem. The recursive problem depends on the inputs and takes a long time to compute. However, one way to speed it up is to use a better guess. Unfortunately, it is not easy to come up with this guess. So what I am doing now is to come up with some "reasonable" guess the first time and then each time the solver calls the function I save the final result of the recursive problem and load it as the guess for the next time the function is called by fsolve. Basicaly, the code I am using has the following structure
fsolve(@my_fun,x0,options)
output = my_fun(param)
if exist('guess_Amat.mat','file')==2
load guess_Amat
end
Amat = my_fun_2(param,guess_Amat) % Compute Amat (a recursive problem where guess_Amat is the initial guess)
% Compute other things using Amat including output variable
% saving Amat
guess_Amat = Amat;
save guess_Amat guess_Amat
end
This works well when I am using a single core. However, when I set in the options
optimoptions('fsolve','UseParallel',true)
the code breaks since (as I understand) different workers are trying to save/load the Amat.mat file at the same time.
My question: Is there a way to adjust the code so that I can "save" A_mat each time and then "load" it for the next evaluation of my_fun to speed up the recursive protion of the code. When using the above code with a single worker loading/saving Amat saves me about 2/3 of the time so I am wondering if there is a way to adapt this appraoch when using the optimoptions('fsolve','UseParallel',true).
0 个评论
采纳的回答
Peter O
2020-9-1
编辑:Peter O
2020-9-1
File access with parallel workers can be tricky, and in the past I've been successful by creating a separate temp directory per worker. If you don't need file access explicitly, and it sounds like you don't, consider using a persistent variable? If the function runs locally on a worker, I believe they'll each hold their own value here. To clear the persistent variable on workers, you need to call pctRunOnAll clear persistent
Since fsolve uses parallel workers to compute the gradient, I'm not sure if there are restrictions on when/how the parallel workers are used (don't have the toolbox, so can't test). Potential code could look like:
fsolve(@my_fun,x0,options)
function output = my_fun(param)
persistent A_guess
if isempty(A_guess)
A_guess = zeros(1000,1); %Initialize
end
Amat = my_fun_2(param,A_guess) % Compute Amat (a recursive problem where guess_Amat is the initial guess)
% Compute other things using Amat including output variable
A_guess = Amat; % Update persistent
end
更多回答(1 个)
Matt J
2020-9-1
编辑:Matt J
2020-9-1
This might work. Basically, I am saying that you must repeatedly call fsolve in a loop having it do 1 iteration at a time.You generate guess_Amat once before the loop and at the end of each loop iteration.
function main()
guess_Amat=___________; %supply initial guess_Amat
options.UseParallel=true;
options.MaxIterations=1;
for i=1:N
[x0,fval]=fsolve( @(param) my_fun(param, guess_Amat) , x0,options);
guess_Amat = my_fun_2(x0,guess_Amat);
end
end
function fval = my_fun(param, Amat)
fval=...
end
2 个评论
Matt J
2020-9-1
You could also experiment with options.MaxIterations settings larger than one. This may diminish overhead from making a fresh call to fsolve, but you will be updating guess_Amat less frequently.
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!