I have a file 1x200 struct how can I distribute it to the worker (32 worker)
2 次查看(过去 30 天)
显示 更早的评论
Dear all, I have a file 1x200 struct, I would like to send it to the cluster (parallel computation), how can I distribute it to the worker (32 worker). I can make it one by one, but it is really take very long time.
Thanks in advance.
Best regards, ahmed
2 个评论
Walter Roberson
2018-1-19
Is it a file (text or binary) or is it just a struct()? Do the workers need access to all of it or only to the portion they are acting on?
采纳的回答
Walter Roberson
2018-1-19
Just create the struct normally before the parfor() loop, and inside the parfor() loop, index the struct by the parfor variable. MATLAB will automatically distribute only the needed portion to each worker.
3 个评论
Walter Roberson
2018-1-22
data = struct('test', num2cell(rand(10,200),1) );
Now data is a 1 x 200 struct with one field named test, with each entry data(K).test being a 10 x 1 vector. The details of the above are not important: I just needed to create a 1 x 200 struct to show parfor.
Now:
parpool(32)
sums = zeros(1,length(data));
parfor K = 1 : length(data)
sums(K) = sum( data(K).test );
end
In this example, parfor does not need to copy all of data to every worker: it only copies the portion needed by the worker. Notice that no special code was needed for this: parfor handles it automatically.
This is an example of a "sliced variable", https://www.mathworks.com/help/distcomp/sliced-variable.html
If the use of the elements of the data structure was more complicated, then parfor could decide that the easiest thing to do is to copy all of data to every worker. In that situation, data would be referred to as a "broadcast variable", https://www.mathworks.com/help/distcomp/broadcast-variable.html
更多回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!