Matlab loop (with load/save) is getting slower and slower

6 次查看（过去 30 天）
Michael
Michael 2013-1-24
编辑： vincent caillet 2017-8-21
Hi
I'm running a large code that is basically:
for i = 1:50
load(i.mat, input);
output = myfun(input);
save(i-out.mat, output);
end
I've run "myfun" hundreds of times and it's long-winded but it works fine and is entirely deterministic. The problem which I'm getting is new, basically since I've added the load/save part, which replaces me just saving the output as one element in a pre-allocated vector. The time taken to execute the loop is getting longer and longer as i increases. The first 4 are completed within hours, the but by the time I reach 20 it's taking a full 24 hours before 3 iterations are completed.
There is no accumulation, each iteration should be totally independent, there is no correlation between the size of i and the input file i.mat. I would test using "for i = 25:1" to make sure order doesn't matter, but I don't have another week.
Does anyone know a simple explanation of why saving/loading (these are large files) might slow this down?
Thanks for any help Mike
  5 个评论
显示 3更早的评论
Michael
Michael 2013-1-24
The .mat files contain a handful of variables, all matrices to double precision, a couple are about 50x4 and a couple are about 200x200. The output are two matrices, 200x200.
Image Analyst
Image Analyst 2013-1-24
编辑：Image Analyst 2013-1-24
Let's see the actual line where you call save. The paraphrased/pseudocode line you gave is not helpful enough.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（3 个）

Image Analyst
Image Analyst 2013-1-24
Might not make any difference but I'd delete any existing mat file before saving to it to make sure it's not adding/appending to it. See if that helps. Use tic and toc to find out which is taking longer and longer - the load() or the save().
  2 个评论
Michael
Michael 2013-1-24
Actually I am overwriting old files of the same name, I'll give this a try and see if it helps, thanks.
Image Analyst
Image Analyst 2013-1-24
Are you running with any breakpoints set? That can really slow things down.

请先登录，再进行评论。

Shashank Prasanna
Shashank Prasanna 2013-1-24
编辑：Shashank Prasanna 2013-1-24
Michael, try the following: put a memory command in the the loop:
for i = 1:50
load(i.mat, input);
output = myfun(input);
save(i-out.mat, output);
memory
end
Either paste the output or let us know if the memory is increasing at each iteration, this may be a leak. Or I can think of JIT issues.
vincent caillet
vincent caillet 2017-8-21
编辑：vincent caillet 2017-8-21
Hi,
I may have a solution for your problem. In your loop, try to use close all force.
My work is essentially to read a large number of images, do some filtering, and save them back. When I loop through a large number of images, the computer memory linearly increases. I've tried to clear as much memory as possible as the end of the loop, but was not very successful.
I notice that clearing the variables AND closing worked however. Not sure if that applies to other people, but for me it's works perfectly. See the example below:
for i = 1:50
load(i.mat, input);
output = myfun(input);
save(i-out.mat, output);
clearvars -expect i OtherVariableOfInterest
close all force
end
While the code is running, inspect your computer performances (Alt+Esc) to see whether the memory keeps increases, or also drops at the end of your loop.

类别

在 Help Center 和 File Exchange 中查找有关 3-D Scene Control 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by