Using 'load' under parfor loop

Question

Anjali 2015-9-30

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/245984-using-load-under-parfor-loop

评论： Walter Roberson 2022-7-12

Hi there,

I am trying to load a variable (E) under my parfor loop. But it is giving me an error

An UndefinedFunction error was thrown on the workers for 'E'.  This might be because the file containing 'E' is not
accessible on the workers.  Use addAttachedFiles(pool, files) to specify the required files to be attached.  See the
documentation for 'parallel.Pool/addAttachedFiles' for more details.
Caused by:
    Undefined function or variable 'E'.

I load using a seperate function as directed in http://ch.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox for saving. Can I please know if it is not possible to load under parfor?

Thanks.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2015-9-30

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/245984-using-load-under-parfor-loop#answer_194250

It is possible to load(), but you have to addAttachedFiles() the file to the pool.

The workers operate in something that functions much like a virtual environment, which only have access to the routines that are directly called (static analysis of calls), and to the routines and data files which have been specifically passed in by attaching them.

3 个评论
显示 1更早的评论隐藏 1更早的评论

Rachel Chen 2020-2-27

编辑：Rachel Chen 2020-2-27

在 MATLAB Online 中打开

When do I add addAttachedFiles() ? I tried:

1. Adding it just after starting parfor

load('my_variables.mat');                                   % variables a and b
myMatrix = zeros(length(a), length(b));
for i = 1:length(a)
    parfor j = 1:length(b)
        poolobj = gcp;
        addAttachedFiles(poolobj, {'my_variables.mat'});    
        %load('my_variables.mat')
        hi = a(i)*10 + b(j)*10^2;
        myMatrix(i,j) = hi;
    end
end

2. Adding it before starting parfor

load('my_variables.mat');                                   % variables a and b
myMatrix = zeros(length(a), length(b));
for i = 1:length(a)
    poolobj = gcp;
    addAttachedFiles(poolobj, {'my_variables.mat'});        
    parfor j = 1:length(b)
        hi = a(i)*10 + b(j)*10^2;
        myMatrix(i,j) = hi;
    end
end

But both didn't work.

Method 1 throws out the error using

parallel.Pool.addAttachedFiles (line 23) - Expecting input number 1, pool, to be non-empty. 

Method 2 displayed "

Analysing and transferring files to the workers....done

. BUT it threw out the exact same issue OP brought out.

What should I do? If it helps, the parfor loop is inside a function which is called by another MATLAB code

Makiko Suitani 2022-7-12

I think you should call the external function and load it in the external function.

parfor j = 1:length(b)

[ x y z ] =load_func('my_variables.mat')

…

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function [ x y z ] =load_func(file)

load( file );

Walter Roberson 2022-7-12

在 MATLAB Online 中打开

Always use the function form of load, which returns a structure with one field for every variable loaded. Then extract information from the struct into the variables.

function [ x y z ] =load_func(file)
load( file );

will not establish any connection between variables x y z stored in the file, and what is to be returned.

function [ x y z ] =load_func(file)
S = load( file );
x = S.x;
y = S.y;
z = S.z;

请先登录，再进行评论。

Using 'load' under parfor loop

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

3 个评论
显示 1更早的评论隐藏 1更早的评论

另请参阅

类别

标签

Community Treasure Hunt

Using 'load' under parfor loop

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

3 个评论 显示 1更早的评论隐藏 1更早的评论

另请参阅

类别

标签

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

3 个评论
显示 1更早的评论隐藏 1更早的评论