Reliably determine RAM consumption of MATLAB variables
20 次查看(过去 30 天)
显示 更早的评论
Matt J
2024-2-29
I am looking for a reliable way to determine how much RAM is devoted to Matlab variable data at a given moment. The whos() command does not work, because it doesn't account for copy-on-write sharing. For example, this code results in 2MB of Matlab data, but whos() counts it as 4MB,
A=rand(512);
B=A;
whos A B
Name Size Bytes Class Attributes
A 512x512 2097152 double
B 512x512 2097152 double
Likewise the memory() command is not reliable, because it requests a memory tally from the operating system, whose estimate can fluctuate from moment to moment depending on what the operating system is doing in the background.
There must be a way to do this, right?
14 个评论
Steven Lord
2024-2-29
I am looking for a reliable way to determine how much RAM is devoted to Matlab variable data at a given moment.
How would you plan to use that information were it available?
Matt J
2024-2-29
编辑:Matt J
2024-2-29
@Steven Lord Nothing beyond the obvious. It would allow me to measure how much my code, or sections of it, are consuming and thereby pursue more optimal memory usage.
John D'Errico
2024-2-29
Why must there be? In a system where the actual memory taken up by B there will change, and in a way that is not fully under your control, you don't want to rely on it being less. I'd just say if you are living that close to the edge, where the size of a variable matters, then you need to get more memeory, and at the same time, look for better ways to do what you are doing.
Matt J
2024-2-29
编辑:Matt J
2024-2-29
@John D'Errico Why would the memory taken up by B not be fully under my control? As the coder, I'm the one who manipulates B.
Matt J
2024-2-29
编辑:Matt J
2024-2-29
Perhaps as further motivation, I would point out that whos() doesn't even reliably report the RAM consumption of a single isolated variable. Below, whos() thinks that F consumes 4 MB, but we know from my initial post above that a 512x512 double matrix only consumes 2 MB (cf. also, this thread). Surely, the grid vectors and other incidental data in a griddedInterpolant object are second order in RAM consumption to the F.Values property.
F=griddedInterpolant(rand(512));
whos F
Name Size Bytes Class Attributes
F 1x1 4202736 griddedInterpolant
John D'Errico
2024-2-29
编辑:John D'Errico
2024-2-29
Yes, you do control it, sort of. I knew this would be a point of contention. At the same time though, MATLAB is the one making the decision, and you need to live with how it makes that choice.
My point is still, assume a variable takes up as much memory as it would take if fully fleshed out, and you will be safe. Second, to avoid living that close to the edge. You will fall off the cliff far too easily.
Walter Roberson
2024-2-29
The size required to represent griddedInterpolant varies with a complicated formula in the number of rows and columns. 4202736 is not twice 2097152.
Matt J
2024-2-29
编辑:Matt J
2024-2-29
The size required to represent griddedInterpolant varies with a complicated formula in the number of rows and columns. 4202736 is not twice 2097152.
It's pretty close:
4202736/2097152
ans = 2.0040
As I said, it is understood that there is other property data in a griddedInterpolant object that accounts for the difference from a perfect factor of two, but it is second order.
Here is another way to see that whos() is double-counting the memory consumption of F.Values for some reason. Below, the agreement between m1 and m2 clearly shows that V and F consume approximately the same memory, but whos() thinks it is twice that.
clear
M0=memory().MemUsedMATLAB/2^30; %baseline memory
V=rand(1e4);
m1=memory().MemUsedMATLAB/2^30 - M0 %consumption of V only
F=griddedInterpolant(V);
clear V;
m2=memory().MemUsedMATLAB/2^30 - M0 %consumption of F only
m3=whos('F').bytes/2^30 %consumption of F according to whos()
m1 =
0.7426
m2 =
0.7426
m3 =
1.4903
Matt J
2024-2-29
编辑:Matt J
2024-2-29
My point is still, assume a variable takes up as much memory as it would take if fully fleshed out, and you will be safe.
I feel like that's asking too much of the coder. The issue I am talking about applies beyond the simple example of two value-semantic copies that I presented. If A and B were handle objects (some people do a lot of work with those), I am supposed to be safe in the assumption that they will always share memory. And in a more serious example, I could have hundreds of handle-semantic copies all over the place. It would be way overconservative to have to pretend that N handle-semantic copies consume O(N) memory.
Steven Lord
2024-2-29
Since we're talking about objects, suppose I had a class with one public property and fifty private properties. Would you expect whos to show you the memory consumed by all fifty-one properties (thus exposing at least some information about those private implementation details) or would you expect it to just show the memory consumed by the one public property?
How about if this was a Java object? How much memory should MATLAB report?
P = java.lang.Double(pi)
P =
3.141592653589793
whos doesn't even try in this case.
whos P
Name Size Bytes Class Attributes
P 1x1 java.lang.Double
As other examples with some questions about your expected ideal behavior, since you referred to memory sharing:
clear all
A = ones(100);
B = A;
whos A B
Name Size Bytes Class Attributes
A 100x100 80000 double
B 100x100 80000 double
Would you expect this to show all the memory as belonging to A (with B listing 0 bytes), all the memory as belonging to B (with A listing 0 bytes), half to each, etc.? Why did you answer that way?
A = 0;
whos A B
Name Size Bytes Class Attributes
A 1x1 8 double
B 100x100 80000 double
If all the memory had belonged to A in the previous example, with B having been listed as having 0 unique bytes, would you be surprised by the sudden apparent "spike" in B's memory usage despite it not being changed or even referred to between the two whos calls? [If your previous question had listed B as owning all the memory, set B to 0 instead and use "B = A" in the next example.]
A = B;
whos A B
Name Size Bytes Class Attributes
A 100x100 80000 double
B 100x100 80000 double
Does A "reclaim its memory" (as listed in whos) from B, or is it now a copy of B and so consume 0 bytes? Why?
C = ones(100);
D = C;
C = D;
whos C D
Name Size Bytes Class Attributes
C 100x100 80000 double
D 100x100 80000 double
How much memory does C claim to own and how much memory does D claim? Should they claim "joint custody" and split the difference?
Matt J
2024-2-29
编辑:Matt J
2024-2-29
@Steven Lord I would be happy if the memory() command offered a total and accurate tally of all RAM consumed by variables, something like,
consumed = memory().MemUsedByMATLABVariables
It already tries to do this anyway, but because it uses the background OS, doesn't always get accurate or stable results.
Walter Roberson
2024-2-29
编辑:Walter Roberson
2024-2-29
for rectangular double matrices of size n by n, griddedInterpolant occupies 16*n*(n + 1) + 240 bytes
F = griddedInterpolant(rand(10));
struct(F)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be avoided. Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
ans = struct with fields:
GridVectors: {[1 2 3 4 5 6 7 8 9 10] [1 2 3 4 5 6 7 8 9 10]}
Values: [10×10 double]
Method: 'linear'
ExtrapolationMethod: 'linear'
ValuesForCopy: [10×10 double]
UnderlyingInterpolantForCopy: []
I don't understand why ValuesForCopy exists, but it does...
The point is that griddedInterpolant "legitimately" takes up twice as much memory as the input array, and whos() does report its memory usage correctly.
Matt J
2024-2-29
编辑:Matt J
2024-3-1
@Walter Roberson I don't know if you're referring to a doc, but it still doesn't resolve the contradiction I presented to you in my previous post. According to your formula, m3 is correct, but that means memory() gave unreliable results. Which do we trust, and why?
Matt J
2024-3-1
编辑:Matt J
2024-3-1
@Walter Roberson the fact that ValuesForCopy exists doesn't prove that griddedInterpolant legitimately takes up twice as much memory. They could be shallow copies of one another. In fact, it seems the likely explanation for why memory() is a factor of 2 in disagreement with whos().
回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Numeric Types 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!发生错误
由于页面发生更改,无法完成操作。请重新加载页面以查看其更新后的状态。
您也可以从以下列表中选择网站:
如何获得最佳网站性能
选择中国网站(中文或英文)以获得最佳网站性能。其他 MathWorks 国家/地区网站并未针对您所在位置的访问进行优化。
美洲
- América Latina (Español)
- Canada (English)
- United States (English)
欧洲
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
亚太
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)