Cluster with 8 workers runs only one job at a time

3 次查看（过去 30 天）
Stephen
Stephen 2014-6-18
评论： Stephen 2014-6-19
I am using a local cluster (parcluster) to run 6 jobs. When I examine my cluster, I see that it reports 8 workers. Since I'm submitting the 6 jobs simulataneously, I would expect that all six would start running right away. Instead, the jobs are only executed one at a time with the others remaining in a 'queued' state. Is there a way to run more than one job at a time, or (best case) run all jobs simultaneously?
  2 个评论
Edric Ellis
Edric Ellis 2014-6-18
How are you submitting these jobs? I just tried this using the BATCH command, and saw this:
>> c = parcluster('local');
>> for idx = 1:6, batch(c, 'pause(10)'); end
>> c.Jobs
ans =
6x1 Job array:
ID Type State FinishTime Username Tasks
------------------------------------------------------------------
1 1 independent running eellis 1
2 2 independent running eellis 1
3 3 independent running eellis 1
4 4 independent running eellis 1
5 5 independent running eellis 1
6 6 independent running eellis 1
Stephen
Stephen 2014-6-18
The jobs are submitted as a group of 20 tasks using createTask, as follows:
myCluster = parcluster;
for i = 1:6
workerTable{i,1} = createJob(myCluster);
for j = 1:20
createTask(workerTable{i,1},@lengthyFxn,7,{inputArgs});
end
submit(workerTable{i,1});
end

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Sean de Wolski
Sean de Wolski 2014-6-18
The tasks are not running serially as you expect but are running in parallel and are thus using all of the resources available. This is why the next job has to wait until the previous has finished running all of its tasks.
  3 个评论
显示 1更早的评论
Stephen
Stephen 2014-6-19
How could I increase the number of workers in the parpool? Would increasing the number of workers help the background processes to finish sooner or is that capped by system resources?

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Job and Task Creation 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by