matlab crashes when using parfor because of low memory

Question

Nils 2015-8-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/235070-matlab-crashes-when-using-parfor-because-of-low-memory

回答： Varun Bhaskar 2015-8-25

Hey everyone,

I have a problem when using parfor in Matlab. I generated a table with about 2.5mio by 1500 parameter which I want save to a postgreSQL Database.

In this table are different parts which I want to save in a parfor loop (It is possible to split this table into a multiple smaller tables). When I use a normal for loop it is running perfectly however with a parfor Matlab is crashing all the time because Matlab duplicates the entire workspace for each worker. Is there any way to control which variables are send to a worker or can I delete unused variables on a worker?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Adam 2015-8-21

It should only copy those variables used within the parfor loop to the workers, but if you do not get the slicing correct it may copy the entire dataset to all workers or the entire results array or whatever else is being shared between the workers.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Varun Bhaskar 2015-8-25

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/235070-matlab-crashes-when-using-parfor-because-of-low-memory#answer_190351

Hi,

Can you have variables that are required by each worker declared after opening parpool, and the ones that needn't be distributed among workers before calling parpool?