About memory manipulation/speed up program

Question

C Zeng 2013-6-13

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/78979-about-memory-manipulation-speed-up-program

Hi, All,

I am doing a large scale computation problem, and I have several variables, which is large scale (>100MB). I was told that MATLAB will not automatically remove the not used memory in running, is that right?

If so, can I put into the code that "clear -var" to remove it, so that speedup the program? Is there some one who has done this before? What is the effect then?

Thanks.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

C Zeng 2013-6-14

Sorry, I may not be very clear on this question. I am doing large scale optimization computation. I am wondering if I remove some unused large variable to make memory efficient, will there a chance to speedup the computing a lot?

Or does someone have done this before? Thanks.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2013-6-14

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/78979-about-memory-manipulation-speed-up-program#answer_88839

There are some situations in which removing unused large variables could speed up calculations a lot, but if the variables are just sitting in memory without being touched then probably it is not significantly affecting the speed of your calculations.

What hammers speed the most in MATLAB tends to be failing to pre-allocate arrays to reasonable sizes, and then to increase the size of the array on (most or every) iteration.

6 个评论
显示 4更早的评论隐藏 4更早的评论

Walter Roberson 2013-6-14

Smaller memory can make a difference in two possible ways:

1) If, between everything running on the computer (including the operating system), more initialized logical memory is being used than there is physical memory, then the operating system will need to "swap" active memory to and from disk.

When physical memory gets full, and more memory is requested, the operating system would choose a block of physical memory, write the contents of the block to disk, scrub the contents of the block of physical memory, and then turn over the block of physical memory to whichever program had requested the memory. When the original program needs the block of logical memory back, then the operating system would choose some block of physical memory, write it's contents to disk, scrub the physical memory, and then read the block of logical memory off of disk into the block of physical memory and turn the block of physical memory over to the program to continue.

This process of "swapping" can be very very slow compared to the case where there is enough physical memory for everything.

When there is enough physical memory for everything then it does not matter (for this purpose) whether any particular block of physical memory is being actively used or not: this mechanism does not need to kick in until there is more logical memory needed than there is physical memory to store it.

2) When memory is being allocated and deallocated as expressions get calculated and results discarded, each time a block of memory is needed for storing an array, MATLAB has to look through its list of available memory blocks (that are allocated to MATLAB but are idle) and pick a "good one" to hand out. "good" is going to depend upon the pattern of future memory use. Handing out the first available block that is at least as big as is needed can turn out to be a bad strategy. If all of the "idle" variables are cleared out, then MATLAB would have greater flexibility about where to assign new requests, which can speed up the process of finding the "best" block to allocate. So it can make a difference -- but whether it will or not is going to depend on your access patterns and the amount of memory you have available to MATLAB. My testing with my program would not give any useful information about how things would go with your program.

Walter Roberson 2013-6-15

C++ is compiled to machine language. MATLAB is an interpreter. MATLAB Compiler, despite its name, produces an intermediate form that is interpreted. MATLAB Coder processed code is compiled, but MATLAB Coder can only handle a fraction of MATLAB and libraries.

Loops have gotten much better in time in MATLAB.

MATLAB knows how to interface to numeric routines such as BLAS that are highly optimized and potentially multiprocessor (even without the Parallel Computing Toolkit); MATLAB also has some heuristics to decide when it would be faster not to use those numeric libraries. MATLAB is thus potentially quite efficient, depending on what is to be done.

You could write calls to those routines in C++, but it would probably take more work and be less clear. You get a trade-off of development time versus execution time. With fast systems these days, development time is often (but not always) the more expensive over the life-cycle of the program

C Zeng 2013-6-17

Dear Walter, thanks for your comments, I did not mean to hurt Matlab, in stead I like it. I was told that C++ is more faster for large scale problem than MATLAB, and it may be so however the advantage may not be too much.

And yes you are right, the development time is much less in Matlab than C++ and that is why I prefer Matlab.

For the loop, I recently found vectorize the code can largely increase the speed in Matlab. I have not tried parallel toolkit but will do it later. Thanks for your answers!

请先登录，再进行评论。

About memory manipulation/speed up program

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

6 个评论
显示 4更早的评论隐藏 4更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

About memory manipulation/speed up program

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

6 个评论 显示 4更早的评论隐藏 4更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

6 个评论
显示 4更早的评论隐藏 4更早的评论