How to save data from Genetic Algorithm in case MATLAB crashes?

2019 12 16
1 个回答
回答已采纳
更新时间：2022 11 30
7 次查看（30 天）

1 个投票

Hi there,
I'm using the GA to find input parameters for my model. My fitness function is computationally expensive and takes about 2 days. In situation if MATLAB unexpectedly crashes, is there a way to save the data?
I have used the 'OutputFcn' option however this seems to save the data once my MATLAB finish the simulation. I purposely stop MATLAB and re-open it but can't seem to access the saved data.
If anyone have any useful tips to save simulation data while the simulation is running so I can access in case it crashes, I would be very grateful.
Many thanks,
How

10 个评论
显示 8更早的评论 隐藏 8更早的评论

Stephen23
Stephen23 2019-12-16
编辑：Stephen23 2019-12-16
"My fitness function is computationally expensive and takes about 2 days"
Do you mean that your fitness function takes two day to calculate, or the GA does?
"In situation if MATLAB unexpectedly crashes, is there a way to save the data?"
Once it has crashed, not really... but you could regularly save the GA population (e.g. using OutputFcn), and then start the GA again using the latest population to set the InitialPopulationMatrix.
Syahmeer How
Syahmeer How 2019-12-16
编辑：Syahmeer How 2019-12-16
Cheers Stephen for the quick reply. Yup I have read about using the latest population and restarting the simulation again from that population if it stops.
Just wondering though if you could share how to regularly save the population while the GA runs?
Actually I'm using a PSO function which I found from the file exchange but the way it's written was very much like the ga function on matlab, so I wrote ga here for better understanding.
It's the PSO aka (ga) that takes 2 days to calculate. I have written an OutputFcn which saves the population when I can only retrieve the data once the function finished calculating. Just not sure how to write the code in a such a way that if I stop Matlab on purpose, it would give me the output, without waiting for the function to finish calculating. Hope that makes sense.
Cheers
How
Here's my OutputFcn code if it helps and a script which I run the PSO function
OutputFcn code: @gaoutfun
function [state,options,optchanged] = gaoutfun(options,state,flag)
persistent state_record
if isempty(state_record)
state_record = struct('Population', {}, 'xGlobalBest', {}, 'Score', {},'Iteration',{},'SS',{});
end
if nargin == 0
state = state_record;
options = [];
optchanged = [];
else
state_record(end+1) = struct('Population', state.Population, 'xGlobalBest', state.xGlobalBest, 'Score', state.Score,'Iteration',state.Generation,'SS',state.fGlobalBest);
optchanged = false;
end
end
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
% Script I used to run the pso
tic
clear gaoutfun
options=psooptimset('Generations',200,'PopulationSize',40,'PlotFcns',{@psoplotbestf,@psoplotswarmsurf},'OutputFcns',@gaoutfun);
[ypso,fval,exitflag,output,population,scores]=pso(fitnessfcn,nvars,[],[],[],[],lbound,ubound,[],options);
record = gaoutfun();
toc
gapopulationhistory = vertcat(record.Population);
gabesthistory = vertcat(record.xGlobalBest);
gascorehistory = vertcat(record.Score);
gaiterhistory = vertcat(record.Iteration);
gasshistory = vertcat(record.SS);
save lale.mat
gahistory = [gascorehistory, gabesthistory, gapopulationhistory];
Stephen23
Stephen23 2019-12-16
编辑：Stephen23 2019-12-16
Define the outputFcn following the documentation:
https://www.mathworks.com/help/gads/genetic-algorithm-options.html#f17837
https://www.mathworks.com/help/gads/custom-output-function-for-genetic-algorithm.html
Pay careful attention to the flag values, the input arguments, the output arguments. You would need to save the data on every 'init' call, or if that slows down your code too much, then use persistent and a simple counter to save the data every N calls (for an N of your choice).
Syahmeer How
Syahmeer How 2019-12-16
This was the OutputFcn I defined, but will probably need some thinking about calling it on every N calls to save the data..
function [state,options,optchanged] = gaoutfun(options,state,flag)
persistent state_record
if isempty(state_record)
state_record = struct('Population', {}, 'xGlobalBest', {}, 'Score', {},'Iteration',{},'SS',{});
end
if nargin == 0
state = state_record;
options = [];
optchanged = [];
else
state_record(end+1) = struct('Population', state.Population, 'xGlobalBest', state.xGlobalBest, 'Score', state.Score,'Iteration',state.Generation,'SS',state.fGlobalBest);
optchanged = false;
end
end
Star Strider
Star Strider 2019-12-16
It would likely be worth experimenting to see of OutputFcn can include a way of using save to append the Best score for the current generation to a .mat file or text file, likely opening it and closing it each time so that nothing is lost. The documentation states that Best is part of the state structure, so it should be available. The documentation on Custom Output Function for Genetic Algorithm offers guidance.
I have never attempted this with ga (my optimisations rarely take more than a few hours at most), so I have no personal experience with it. (In the past, I wrote my own genetic algorithm codes, and saved the best individual to a text file in each generation.)
Stephen23
Stephen23 2019-12-16
编辑：Stephen23 2019-12-16
@Syahmeer How: I strongly recommend that you use the flag input to initilalize your function, and select when to save the data, and clean up when complete.
When you wrote save I presumes that you actually meant save, as in "export the data to file". If you expect MATLAB to crash or hang, there is little point in your approach of just storing that data in some variable/s. Appending to a .mat file is probably the simplest.
Syahmeer How
Syahmeer How 2019-12-16
@Star Strider: Thanks for your reply, that's exactly what I've been thinking but just not sure how to write it with little examples on the web. I will have to perform some more trial and error to see if I could save the data in a mat.file with each generation as the simulation is running. But would appreciate if you could share some examples about doing that though :)
Syahmeer How
Syahmeer How 2019-12-16
@ Stephen Cobeldick. Cheers for your recommendation Stephen. Yup I see what you mean here. Will have to do a bit more digging to append the data into a .mat file as the simulation is running. Still a bit clueless on how to do that though.
Star Strider
Star Strider 2019-12-16
@Syahmeer How — My pleasure. I have some code that specifically uses ga, so I will experiment with that and will post back here if I have a successful result.
Syahmeer How
Syahmeer How 2019-12-16
@starstrider Thank you so much :) I look forward to hearing back from you

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

Star Strider
Star Strider 2019-12-17

2 个投票

It took a bit of trial and error, however this appears to work:
function [state,options,changed] = SaveOut(options,state,flag)
file_name = 'SaveBest.mat'; % Name File
if strcmp(flag,'iter')
ibest = state.Best(end);
ibest = find(state.Score == ibest,1,'last');
bestx = state.Population(ibest,:);
previous = load('SaveBest.mat');
var = [previous.var; bestx]; % Read Previous Results, Append New Value
save(file_name, 'var') % Write ‘Best Individual’ To File
end
changed = true; % Necessary For Code, Use Appropriate Value
end
It reads the existing saved values of ‘var’, appends the new value to the end of the matrix, then writes the new matrix to the .mat file.
It never occurred to me that this was even possible! I will use it in my longer optimisations from now on.

9 个评论
显示 7更早的评论 隐藏 7更早的评论

Syahmeer How
Syahmeer How 2019-12-17
Cheers for the code StarStrider!! I seem to have an error of
Reference to non-existent field 'var'.
Do I have to change var to something else?
Syahmeer How
Syahmeer How 2019-12-17
I guess it's probably that the simulation I run was a wee bit short so 'var' may not exist at that time. I would run it longer and stop the simulation to see if the works this time.
Thanks so much for your help StarStrider!!
Star Strider
Star Strider 2019-12-17
编辑：Star Strider 2019-12-18
I ran that code so many times to test it that I did not encounter that problem.
My apologies.
I deleted the file first each time during this test, then ran my optimisation again without problems with slightly edited code.
This works:
function [state,options,changed,str] = SaveOut(options,state,flag)
file_name = 'SaveBest.mat'; % Name File
if strcmp(flag,'init')
Var = state.Population;
save(file_name, 'Var') % Write ‘Best Individual’ To File
elseif strcmp(flag,'iter')
ibest = state.Best(end);
ibest = find(state.Score == ibest,1,'last');
bestx = state.Population(ibest,:);
previous = load('SaveBest.mat');
Var = [previous.Var; bestx]; % Read Previous Results, Append New Value
save(file_name, 'Var') % Write ‘Best Individual’ To File
end
changed = true; % Necessary For Cide, Use App
end
If you want to see the last five rows in the .mat file (as I named it):
BestInGen = load('SaveBest.mat')
LastFive = BestInGen.var(end-4:end,:)
EDIT — (18 Dec 2019 at 23:34)
Changed ‘var’ to ‘Var’ to avoid overshadowing the var (variance) function if the file is loaded without output arguments. Code otherwise unchanged.
Syahmeer How
Syahmeer How 2019-12-17
Brilliant!!! The code works! Thanks heaps for your help! Now I don't have to worry about running my simulations for a very long time fearing it might crash in any second haha.
Thanks again and have a very good Christmas break :D
Star Strider
Star Strider 2019-12-17
I very much appreciate your compliment!
As always, my pleasure!
I wish you the best the Christmas Holidays can bring!
Syahmeer How
Syahmeer How 2019-12-17
No worries, and thanks for your wish :)
Actually I have one last question if that's okay. So if in unexpected circumstances my simulation crashes and I managed to save the 'Best scores' of my last generation. Is this the only variable I need to initialise to complete the rest of my simulation?
Star Strider
Star Strider 2019-12-17
As always, my pleasure!
The ‘var’ field in my code would be the InitialPopulationMatrix (or at least part of it). The rest of your optimoptions structure would be as it was previously. The ga funciton will reconstitute everything else it need in the first iteration.
Manjur Basnet
Manjur Basnet 2022-11-30
Hi Star Strider, I have been trying to use the Saveout function to save best value from each generation but I keep getting this error after 1 generation
"Unrecognized field name "Best".
Error in SaveOut (line 7)
ibest = state.Best(end);
Error in gaoutput (line 51)
[state,optnew,changed] = feval(functions{i},options.OutputPlotFcnOptions,state,flag,args{i}{:});
Error in gamultiobjsolve (line 86)
[state,options] = gaoutput(FitnessFcn,options,state,currentState);
Error in gamultiobj (line 338)
[x,fval,exitFlag,output,population,scores,residuals] = gamultiobjsolve(FitnessFcn,nvars, ...
Error in pareto (line 28)
[x,fval,exitFlag,output] = gamultiobj(fun,5,A,B, Aeq, beq, lb,ub,nlcon, intcon, options);"
Can you help me with this? I tried searching for this kind of error but could find nothing.
Thank you so much.
Star Strider
Star Strider 2022-11-30
@Manjur Basnet — The correct thread for your problem is: Saving population individauls for each generation of GA Multiobjective optimisation.
Unfortunately, Shahrbanoo Shamekhi-Amiri didn’t accept it in spite of my posting a working solution.
See if that Answer solves your problem.

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Genetic Algorithm 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by