bizarre SPMD error

9 次查看（过去 30 天）
jay cassano
jay cassano 2011-11-13
I was using Matlab R2011b and had strange SPMD error.
The structure of the code looks like this:
load A.mat % A is super large, e.g. 7GB
spmd
B = ...; C = ... % create some "small" local variable
do something with A,B,C ...
end
Our machine has over 40GB memory.
Here are the problems:
1) The program runs perfectly well in parallel, if A is smaller, e.g 0.5G;
2) The serial version of the program runs perfectly well, if A is as huge as 7GB.
3) spmd reports error if A is huge:
---------------------------------------
Error using distcompserialize
Error during serialization
Error in spmdlang.RemoteSpmdExecutor/initiateComputation (line 82)
fcns = distcompMakeByteBufferHandle( ...
Error in spmdlang.spmd_feval_impl (line 14)
blockExecutor.initiateComputation();
Error in spmd_feval (line 8)
spmdlang.spmd_feval_impl( varargin{:} );
Error in myprogram >(spmd) (line xx)
spmd
-----------------------
so this is a memory issue? But although A is huge, it is shared. My local variables are pretty small, a few MB.
Any idea what went wrong? Thanks.
Yingjie

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2011-11-13
Shared doesn't mean that copies of it do not have to be sent to the different workers (which, after all, might not be on the same host.)
Please see Edric's solution http://www.mathworks.com/matlabcentral/answers/20126-parfor-problems

类别

在 Help Center 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by