Genetic Algorithm + Parallel Computing + Global variables

Question

Jonas 2022-8-27

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1788340-genetic-algorithm-parallel-computing-global-variables

编辑： Matt J 2022-8-29

Hello to all,

I have been able to run a GA in parallel (10 workers in total). Now I want to store in a kind of "result matrix" what happens on each worker. For example the simulated individual, the CPU time, the cost and the number of iterations. I tried defining global variables in the main script, where I just call the ga(). Then in the objective function I call the global variables and assign new values. Unfortunately, I only get one row in my "result matrix".

I also tried parfevalOnAll and in the called function I load and assign the values. But this does not work at all.

Does anyone have any idea how to handle this?

Thanks a lot already!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2022-8-27

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1788340-genetic-algorithm-parallel-computing-global-variables#answer_1035405

编辑：Matt J 2022-8-29

在 MATLAB Online 中打开

If you're using UseVectorized, it should be relatively easy. Within your parfor loop, store the results of processing each population member to a struct array and store that to an externally scoped cell array ResultMatrix as below. Later you can aggregate the different ResultMatrix{i}(j) according to the worker label stored in ResultMatrix{i}(j).labindex.

function main
      ResultMatrix={};
      
      opts=optimoptions('ga','UseVectorized',true);
      
      x=ga(@fitnessFunc,nvars,A,b,Aeq,beq,lb,ub,nonlcon,intcon,opts));
      function out=fitnessFunc(X)
            parfor i=1:size(X,1)
  
             tic;
               %process population members here
             T=toc;
             
                S(i).cpuTime=T;
                S(i).popmember=X(i,:);
                
                ...
                    
                S(i).labindex=getCurrentTask().ID;
                S(i).cost=cost;         
            end
    
           ResultMatrix{end+1}=S;
           
           out=vertcat(S.cost);
        
      end
  
  
end

6 个评论
显示 4更早的评论隐藏 4更早的评论

Jonas 2022-8-27

在 MATLAB Online 中打开

Thank you for your fast answer!

Because I have a very high simulation time, I think I can't use UseVectorized. I got a co-sim, my structure is kind of like this:

%main script
...
    
global result_mat
global n_iter
n_iter=0
result_mat=[0,0,0,0]
    options=optimoptions('ga','PlotFcns',@gaplotbestf, 'Display','iter', ...
    'PopulationSize', PopSize, 'MaxGenerations', MaxGen,...
    'EliteCount', EliteCo,'CrossoverFraction',CrossFra,...
    'MutationFcn', MutFcn,'UseParallel', true, 'UseVectorized', false);
[x,fval,exitflag,output,population,scores] = ga(...
    @(x) object_fcn (x), nvars,[],[],[],[],lb, ...
    ub,[],intcon, options);
...

So I call the objective function "object_fcn".

And the object_fcn is structred like this:

...
    while status_sim ==0 && counter_while_try <4
    try
        tic
        sim(simulink_model)   %within the simulink model there are different fmus
        status_sim=1
        CPU_time=toc
    catch 
        ...  %try again
    end
    end
    ...
        
    run("calc_cost.m")  %Here I use the simulation results to calculate the costs of each simulation
        
    %and know in the end I want to add some variables to the result_matrix
    
    
    global result_matrix
    global n_iter
    
    n_iter=n_iter+1
    
    G_iter=[CPU_time, x, cost, n_iter]
    
    result_matrix=[result_matrix, G_iter]
        

So this is working in general. But in the end my result_matrix is like [0,0,0,0;1231, x, 3.2, 0] and the values of the other parallel simulations aren't there.

Matt J 2022-8-27

编辑：Matt J 2022-8-29

To be honest I don't really understnad what this is excatly doing.

You probably need to read about what UseVectorized does. Basically, it will pass all members of the current population to the objective function at once in a single input matrix X. When you do this, your objective function has the responsibility for computing the cost of all population members X(i,:) in the current generation. You can divide up this work with whatever parallel computing toolbox functions you wish, in the case of my example with parfor. The advantage of this is that your objective function becomes the workspace where the parallelization happens, so you have access to it and can save the results to external variables like ResultsMatrix.

in my case, the ga is calling the objective function in parallel. So I don't have a parfor loop in my function. Otherwise, your code would solve my problem.

Yes, but the whole point of this forum is that we can suggest you stop doing what you're doing and do something else that might better.

Jonas 2022-8-28

Thank you for the explanation.

Yes, but the whole point of this forum is that we can suggest you stop doing what you're doing and do something else that might better.

You are absolutely right. Since yesterday I tried to change the structure and use vectorized. I was not able to do it. Since I am running out of time, I will stick with my structure.

Do you or anyone else have any other ideas for a workaround? It doesn't have to be the prettiest solution.

Walter Roberson 2022-8-28

global variables never transfer between parallel workers or workers and client.

请先登录，再进行评论。

Answer 2

Matt J 2022-8-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1788340-genetic-algorithm-parallel-computing-global-variables#answer_1036180

编辑：Matt J 2022-8-28

在 MATLAB Online 中打开

Do you or anyone else have any other ideas for a workaround? It doesn't have to be the prettiest solution.

You might be able to accumulate the results to a .mat file, as opposed to a global variable. In your fitness function, you would have something like,

filename="outfile"+getCurrentTask().ID; %worker-dependent temp file
result_matrix=[load(filename).result_matrix, result_matrix];
save(filename,'result_matrix');

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Matt J 2022-8-28

You could also do the same kind of thing in a custom Output Function to add things like the current iteration number to the temp files.

请先登录，再进行评论。

Genetic Algorithm + Parallel Computing + Global variables

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（2 个）

6 个评论
显示 4更早的评论隐藏 4更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Genetic Algorithm + Parallel Computing + Global variables

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（2 个）

6 个评论 显示 4更早的评论隐藏 4更早的评论

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

6 个评论
显示 4更早的评论隐藏 4更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论