parallel computing fitness function inside parallel optimization

5 次查看（过去 30 天）
Rui
Rui 2023-8-29
评论： Sam Marshalik 2023-9-1
I have a calculation intensive fitness function to optimize in genetic optimization.
I plan to use matlab parallel server with 10 compute nodes, each on has 24core. Can I run each objFunction parallel in GA and parallel in each nodes?
Following is example code and target
  • 40 poulation,
  • UseParallel On in GA options
  • Can assign each objFuntion evaluation to indivadual 24core compute and use another parfor inside objFunction?
  • Therefore, it allowed me run 10 parallel objFunction on 10 node. Each objfunction run parallel in each node.
Thanks
% parallel on and 40 population
options = optimoptions('UseParallel',true,'PopulationSize',40);
[x,fval,exitflag,output] = gamultiobj(@objFunction,20,[],[],[],[],lb,ub,options);
%% fitness function required to run parallel
function output = objFunction(x)
a = 1:10000;
output = zeros(1,numel(a));
parfor i = 1:numel(a)
output(i) = a(i) + mean(x);
end
end

请先登录，再回答此问题。

采纳的回答

Walter Roberson
Walter Roberson 2023-8-29
Inside a parallel worker you can vectorize, but not run anything in parallel.
You would need to run multiple matlab sessions, each taking on part of the task.
  5 个评论
显示 3更早的评论
Rui
Rui 2023-8-31
Hi Sam,
Thanks for suggestion for option of NumThreads. Can I run parallel in each worker during optmization? My finitness function is computing intensive. for example,
I understand matlab optimization can do parallel, by default, each worker use 1 core and non-parallel inside worker.
options = optimoptions('UseParallel',true);
[x,fval,exitflag,output] = gamultiobj(@objFunction,20,[],[],[],[],lb,ub,options);
What I am looking for is parallel inside each worker (each worker is one computer node) with MATLAB Parallel Server. For example (each computer node has 4 core).
Worker 1 (compute node 1): 4 core parallel parfor
Worker 2 (compute node 2): 4 core parallel parfor
Worker 3 (compute node 3): 4 core parallel parfor
....
Sam Marshalik
Sam Marshalik 2023-9-1
Hey Rui,
It is a bit tricky, but you can do this.
I first wanted to make sure that gamultiobj could run in a Threaded environment. I confirmed that it is possible by startung a local Thread Pool and running an example there. This means that a MATLAB Parallel Server process worker will be able to start a Threaded pool of workers and run the optimization problem in parallel using Threads.
The next question is how to actually offload this to the cluster. Can you let me know what scheduler you are using? Depending on what scheduler you are using the answer will be different.
I think using batch jobs will be a good way forward, like the below. Each batch job will start a single process worker and then start X number of Threads. The detail to be worked out is how to request the right resources from ths scheduler. You will want to ask for a total of X + 1 cores (1 for the MATLAB Parallel Server worker and X for the Threads).
c = parcluster('myCluster');
c.NumThreads = X;
batchJob = batch(p,@optimFunc, c.NumThreads)
function optimFunc(NumThreads)
parpool("Threads", NumThreads)
rng default % For reproducibility
M = diag([-1 -1]) + randn(2)/4; % Two problem variables
fun = @(x)[(x').^2 / 30 + M*x']; % Two objectives
intcon = 2;
lb = [0 0];
ub = [100 50];
options = optimoptions("gamultiobj","PlotFcn","gaplotpareto",...
"PopulationSize",100, 'UseParallel', true);
nvars = 2;
A = [];
b = [];
Aeq = [];
beq = [];
nonlcon = [];
[x,fval] = gamultiobj(fun,nvars,A,b,Aeq,beq,lb,ub,nonlcon,intcon,options);
end

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Third-Party Cluster Configuration 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by