Parfeval memory leak? Clear memory on parallel workers?

7 次查看（过去 30 天）
Matthias
Matthias 2014-12-9
评论： Francis Plamondon 2020-11-18
Hi,
I have some sort of memory leak when using parfeval.
I have a function that starts a number of parfeval-computations and then uses fetchNext in a loop to retrieve all results.
The memory taken up by the Matlab instances in my parallel pool grows with each execution of the function that uses parfeval.
My question: Am I doing something wrong? Do I have to explicitly clear the memory of the workers after using parfeval? How do I do this? I tried "pctRunOnAll clear all" but it did not free up any memory on the workers (just the client instance).
Right now, the only way I can clear this memory is to shut down and re-start my parallel pool, but that's not a feasible solution for my application.
Thanks, Matthias
  3 个评论
显示 1更早的评论
Atakan Atay
Atakan Atay 2020-3-25
编辑：Atakan Atay 2020-3-25
Hi Matthias,
Currently I experience the very same problem. How did you manage to solve it? I would really appreciate your help.
Thanks for your time.
Francis Plamondon
Francis Plamondon 2020-11-18
Hello,
I also recently struggled with parfeval. I would get 'out of memory' errors after several hours of running a huge batch of jobs.
Here is what I found out: each parallel.FevalFuture object holds a Diary property that contains everything that was displayed in the console window for each job. Since I had a function use disp() for monitoring purpose, it was being recorded in the Diary variable, eventually filling all the memory.
After commenting out the disp function, the memory stopped slowly increasing.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Edric Ellis
Edric Ellis 2014-12-15
Please could you try applying the workaround from this bug report. (The bug report mentions mapreduce, but it includes a fix that affects only parfeval).
  1 个评论
Seth
Seth 2015-2-13
I have a similar problem and tried using this patch and now get errors indicating "no method 'createInstance' with matching signatures found for class 'com.mathworks.toolboox.distcomp.objects.EntityFileProxy'
Do you have any other suggestions? I was running two workers with PARFEVAL and seeing 3gb increase in RAM used each call to them. (The PARFEVAL commands are in a loop)
Thanks.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Startup and Shutdown 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by