Worker can't find attached files

Question

Dominik Schäfer 2020-12-3

2
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/675983-worker-can-t-find-attached-files

评论： Maria 2021-10-6

Hi Matlab Community,

We've set up a Jenkins server for our simulation code. We have a set of test cases, where output data of our code is compared to reference values from the literature or stable versions of the code. Matlab on jenkins server is Version 2020b/Linux.

One test case, which is going into a parfor loop, can't seem to find the source code files for the workers. Other test cases, which run basically the same parts of the code including this exact parfor loop, have no problems whatsoever. I get the following error:

Error using Driver_VLEmix (line 167)

The source code

(*********************************************/Driver_VLEmix.m) for

the parfor-loop that is trying to execute on the worker could not be found.

Auto attached files are being stored in the home directory of the Jenkins user. The file is on the other hand listed via listAutoAttachedFiles(gcp())

Files automatically added to the parallel pool:

*********************************************/Driver_VLEmix.m

I tried a run, where I manually attach the file, still the same result.

This error only occurs on the jenkins server. On our local machines, it works just fine (Linux 2019b, Win 2020b).

This is the only case, where I get the following message when the parpool is initialized:

Analyzing and transferring files to the workers ...done.

Which I'd guess is where the files are automatically attached. Any ideas what could be wrong?

Best regards,

Dominik

3 个评论
显示 1更早的评论隐藏 1更早的评论

Dominik Schäfer 2020-12-3

编辑：Dominik Schäfer 2020-12-3

Hi Edric,

we have basically one main script and one subdirectory src/ containing all functions. At the start of the main script, the subdirectory src/ with all functions is added to the path list.

fetchOutputs(parfeval(@which, 1, 'Driver_VLEmix')) has given:

>> fetchOutputs(parfeval(@which, 1, 'Driver_VLEmix'))
ans =
    '/tmp/tp430c38ea_816a_4e76_9cb2_238c7e8ab218rp38541/a/tp29d74774_ee86_4d86_a833_26b886057d52/Driver_VLEmix.m'

I have checked the list of auto attached files and not all files in src/ have been auto attached. But I guess, the parpool initializer only auto attaches files, that it considers relevant to the running job. I already did a run, where I manually attached the whole src/ directory to the processpool.

Neither eval nor feval used in Driver_VLEmix.m.

Best regards,

Dominik

PeymanS 2021-3-17

Hi Dominik,

I have faced the same problem. Have you found the solution?

Best,

Peyman

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Dominik Schäfer 2021-3-17

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/675983-worker-can-t-find-attached-files#answer_650402

Hi Peyman,

one of the subfunctions in the parfor loop called the (not recommended) nansum function.

When we changed nansum to sum(A,'omitnan') everything worked as intended.

Hope this helps!

Best regards,

Dominik

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

Chien-Han Su 2021-7-1

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/675983-worker-can-t-find-attached-files#answer_737743

编辑：Chien-Han Su 2021-7-1

I accidentally triggered this error as well, and the fail is resulted from, in the parfor loop, an wrongly called non-existent variable/function due to typos.

I think for any one who has the same problem, one possible way to deal this is turn off the parallel procedure at first (using "for" instead of "parfor"). In this way, matlab will return a more detailed error massage, and that's how I figure it out.