Issue with launching Parallel Workers when using TORQUE or PBS

Question

Mohammad Abouali 2016-12-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/315180-issue-with-launching-parallel-workers-when-using-torque-or-pbs

评论： Mohammad Abouali 2016-12-5

采纳的回答： Edric Ellis

Hi,

I am attaching the full output of MATLAB when trying to set useParallel option of GA to true.

This been causing some trouble in launching parallel pool.

Note that I am scheduling 100s of such runs on a computer cluster using Torque (or PBS, well I use qsub command).

So, I am not running them interactively. I am attaching both the full MATLAB Output and the scheduler script that I use to submit the job using qsub.

Any help is appreciated.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Edric Ellis 2016-12-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/315180-issue-with-launching-parallel-workers-when-using-torque-or-pbs#answer_245983

在 MATLAB Online 中打开

One problem might be that the job storage locations are colliding, and you're ending up with many processes trying to write data to the same location. You could work around this by creating a local cluster instance using a unique job storage location. Something like this:

tempLoc = tempname;
mkdir(tempLoc);
clus = parallel.cluster.Local('JobStorageLocation', tempLoc);
parpool(clus);

2 个评论
显示无隐藏无

Mohammad Abouali 2016-12-5

Thanks Edric. I am also suspecting that this is what might be happening, i.e. two different scheduled jobs trying to use same storage locations.

I check the solution you have provided and get back to you if it resolved the issue.

Thank you again.

Mohammad Abouali 2016-12-5

Your suggestion helped that problem. But then some of the runs had problem to execute the GA-Objective function.

One thing that I noticed is that I am requesting nodes=1:ppn=4. But then I launch 4 workers. That makes total of 5 process, (1 main matlab + 4 parallel workers).

I rescheduled the jobs using the work around that you provided and also increased the ppn to 5. So far things seems going fine.

So, I think we got the issue resolved. I am going to accept the answer at this point. and if later I found out it is not working I will post another question.

Thank you so much for your help. I really appreciate it.

请先登录，再进行评论。