system command in parfor loop

2 次查看（过去 30 天）
Sujatha
Sujatha 2022-1-17
评论： Walter Roberson 2022-1-26
I am invoking a C++ executable from the Matlab command prompt.
Interactively, it goes like this:
>> [status,cmdout] = system('multi.exe','-echo')
Input unique file identifier. Ex: 2~04-13-013939 - 2~11-30-091444
And this works fine.
However, I need to invoke the C++ executable multiple times. So I created a parfor loop, like so:
parpool
tic
parfor i=1:nfiles
[status,cmdout] = system('multi.exe','-echo')
str3{i,1}
end
where str3 is a series of inputs:
str3 =
14×1 cell array
{'1~11-24-054646'}
{'1~11-24-095017'}
{'1~11-29-060908'}
{'1~11-29-101743'}
{'1~11-30-091225'}
{'2~11-24-055554'}
{'2~11-24-100903'}
{'2~11-29-060541'}
{'2~11-29-101814'}
{'2~11-30-091444'}
{'3~11-24-055933'}
{'3~11-29-101847'}
{'4~11-24-060505'}
{'4~11-29-101935'}
This ran for over 54 hours, but the task never completed, and I eventually hit Ctrl-C.
Any thoughts/ feedback?
  2 个评论
Mario Malic
Mario Malic 2022-1-17
Check this out. Parfor worked fine for me using Sysem.Diagnostics.Process.
https://www.mathworks.com/matlabcentral/answers/583955-get-the-status-using-system-command-when-program-has-been-closed#answer_485480
In order to relay the messages from workers in parpool, there are functions which I don't remember the name exactly. Check out this example https://www.mathworks.com/help/parallel-computing/parallel.pool.dataqueue.html
jessupj
jessupj 2022-1-17
编辑：jessupj 2022-1-17
just to play devil's advocate, i've approached a similar problem in the past by calling parallel instances of matlab from the shell. it was by no means the most I/O-efficient method and wasted a lot of time opening and closing matlab & pooling. the individual system jobs were geophysical models that took O(days) so the data importing and pool opening overhead was comparatively insignificant.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2022-1-17
parpool
tic
parfor i=1:nfiles
cmd = sprintf('echo "%s" | multi.exe', str3{i,1});
[status,cmdout] = system(cmd, '-echo')
end
  12 个评论
显示 10更早的评论
Sujatha
Sujatha 2022-1-26
addAttachedFiles(ans, tempfile), didn't help.
Walter Roberson
Walter Roberson 2022-1-26
parpool
tic
parfor i=1:nfiles
tname{i} = tempname();
[fid, msg] = fopen(tname{i});
if fid < 0
error('iteration %d failed to create temporary file "%s" because "%s"', i, tname{i}, msg);
end
fprintf(fid, '%s\n\n', str3{i,1});
fclose(fid)
cmd = sprintf('multi.exe < "%s"', tname{i});
[status,cmdout] = system(cmd, '-echo')
end

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Parallel Computing Fundamentals 的更多信息

标签

产品


版本

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by