Memory grows using parallel

DavidePiccinini

2022 10 24

0 个回答

6 次查看（30 天）

0 个投票

Hi, It's my first approach to the parallel computing, and I wrote a matlab code to take advantage this feature in Matlab. My problem is that the memory usage during the parallel run (using a parfor call) grows until I get an out of memory error.

My code is something like:

DATA=rand(1000,1000);

for k=1:1000 
    A=DATA(k,:); 
    B=DATA(k,:); 
    parfor i=1:100 do_something; end 
end

I tried to clear out all the unused variables and pass to the function inside the parfor only the necessary part of data to ensure the smallest overhead, but it seems that in some way the code occupy even more memory for each run. Furthermore the whos output during each run didn't reveal the puzzle (all the variables have the same size).

Any ideas/suggestions to help me solve the problem?

Regards d

5 个评论
显示 3更早的评论隐藏 3更早的评论

DavidePiccinini 2022-10-25

Yes you are right. I'm sorry but the code is a bit more complex, it was an attempt to simplify it.

do_something is a function where A and B are cross-correlated (using xcorr).

The cross-correlation results are added to a text file (one for each A-B couple).

Once the for loop is ended the text files are then readed, processed and deleted (I'm planning to use the memmapfile function for this part).

Alvaro 2023-1-25

编辑：Alvaro 2023-1-25

In this case the code might be too simplified to figure out what the issue is, could you post more of the actual code? It would be good to know how you are writing to that text file and what variables you are using to store the results.

请先登录，再进行评论。

请先登录，再回答此问题。

Follow Question