Batch jobs of similar data size and function files are taking different times for job completion, though all are started at same time.

1 次查看(过去 30 天)
I have a large (n x n) data, which is an input for several dependent function files.
Each element should undergo the same process, and the communication between the elements is not necessary.
So I partitioned my data into 'z' number of zones corresponding to number of cores in local machine.
Three columns and two rows.
r_zones = 2; % row zones
c_zones = 3; % column zones
rz = round(linspace(1,sx,r_zones+1)); %rz = row zones
cz = round(linspace(1,sy,c_zones+1)); %cz = column zones
Six jobs are created for six zones with batch command.
pc = parcluster();
for i = 1:r_zones
for j = 1:c_zones
Xa((rz(i):rz(i+1)-1),((cz(j):cz(j+1)-1))) = 1;
job(jm(i,j)) = batch(pc,@com_sigma,3,{inputs});
end
end
for i = 1:r_zones
for j = 1:c_zones
wait(job(jm(i,j)));
end
end
All the jobs have same size of data ( a difference of one row or column may present) and started at the same time.
But, the finishing time of each job is different and the initial job is taking longer time. And, the CPU utilisation is not more than 70%.

采纳的回答

Raymond Norris
Raymond Norris 2021-8-20
Not sure why the 1st task isn't finishing, but here's another approach. Keep what you have, but rather than using batch, use parfeval. Change
pc = parcluster();
for i = 1:r_zones
for j = 1:c_zones
Xa((rz(i):rz(i+1)-1),((cz(j):cz(j+1)-1))) = 1;
job(jm(i,j)) = batch(pc,@com_sigma,3,{inputs});
end
end
to
pc = parcluster();
pool = gcp('nocreate');
if isempty(pool)
pool = pc.parpool(maxNumCompThreads);
end
for i = 1:r_zones
for j = 1:c_zones
Xa((rz(i):rz(i+1)-1),((cz(j):cz(j+1)-1))) = 1;
job(jm(i,j)) = parfeval(pool,@svd,3,inputs);
end
end
You're start one job (a parallel pool) and then sending individual tasks to it (with parfeval) rather than creating r_zones * c_zones jobs (with batch).

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Startup and Shutdown 的更多信息

产品


版本

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by