Out of memory during parfor

75 次查看（过去 30 天）
Andrea Stevanato
Andrea Stevanato 2018-7-11
评论： Walter Roberson 2020-2-13
I'm trying to execute code like this
for i = 1:100
parfor 1:10
funResult = function(data);
end
end
but I get some error like
Out of Memory during deserialization
or
Out of Memory during serialization
or some workers aborted during execution. I tried this code with different data (smaller of those that cause this error)
How i can solve this problem? I get the error after 8/10 iteration of first for loop.
  8 个评论
显示 6更早的评论
Matt J
Matt J 2018-7-12
编辑：Matt J 2018-7-12
Are you on Windows? The Task Manager will show you how much RAM you are consuming, as the code runs, as well as how occupied all of your cores are.
Andrea Stevanato
Andrea Stevanato 2018-7-13
No, i'm running under Linux and yes I could look how much RAM I'm using.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2018-7-11
编辑：Walter Roberson 2018-7-11
I do not know if it is still the case, but historically there was a limit of 2 gigabytes per array transferred, including on 64 bit versions of MATLAB.
Check your code in case you are using broadcast variables. Sometimes broadcast variables can be rewritten in a way that uses distributed variables instead. For example if two different columns need to be accessed, create two variables, one columns 1:end-1 and the other columns 2:end, and then index each by the parfor index: this will trigger distributed use instead of broadcast use in that situation.
  3 个评论
显示 1更早的评论
Weidong Lin
Weidong Lin 2020-2-13
Hi, have u fixed it? I am facing the same problem.
Walter Roberson
Walter Roberson 2020-2-13
Is MyClass a class variable instead of being numeric? If it is, then I wonder if MATLAB is sending the entire class variable to each worker? With classes, it would not necessarily be correct to subsref on the client and send to the worker. repmat() isn't just a "make this many copies" operator, it is also an implicit cat() operator so the result of repmat is not necessarily 10 different objects: it could be one object with internal "size" 10.
I would have to think more about the circumstances under which it is safe to subref on the client and send to the worker; I can definitely think of some cases where that would get you the wrong answer, or at least behavior that isn't all that well defined (e.g., what if the object is keeping usage stats internally?)

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by