Why is there an out of memory error despite memory() stating there is enough?

2 次查看（过去 30 天）
Jan
Jan 2015-9-25
评论： Steven Lord 2015-9-26
I have the following code:
data(ind) = data(ind) + shift;
data(~ind) = data(~ind) - shift;
The first line crashes with an out of memory error.
ind is a large logical vector and data is a large uint8 vector of the same length as ind. They both have the same size, about 90 MB. According to memory(), the Maximum possible array is 301 MB.
I would like to optimize this, but I do not understand how the indexing works. I would guess that a temporary copy of the data array is created, but why does it not fit in the available memory? Perhaps, there are multiple copies, or the copy is not of type uint8?
  4 个评论
显示 2更早的评论
Steven Lord
Steven Lord 2015-9-25
Does that line crash (which would be bad) or does it throw an error (which is what I would expect if MATLAB can't find a contiguous block of memory large enough?)
Jan
Jan 2015-9-26
Well, it throws an error, and thus crashes my program. It does not crash matlab if that's what you mean.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Walter Roberson
Walter Roberson 2015-9-25
Mixed arithmetic between integer and double involves converting to double, doing the arithmetic and converting back; see http://uk.mathworks.com/matlabcentral/answers/238372-nan-does-not-operate-symmetrically-with-integer-classes
So just uint8(shift) so that you are doing integer/integer arithmetic and you should be okay.
  3 个评论
显示 1更早的评论
Jan
Jan 2015-9-25
That won't always work, for instance if data is a int8, and shift is 150. But that doesn't matter here, I can work around that.

请先登录，再进行评论。

更多回答（1 个）

Steven Lord
Steven Lord 2015-9-25
Keep in mind that arrays in MATLAB need to be stored in contiguous blocks of memory. [I'm just going to deal with numeric arrays; the rules for cell arrays, struct arrays, and objects get a little complicated.]
Even if MEMORY indicates you have enough total memory, you might not have a contiguous block large enough to hold your result. Consider for demonstration purposes a computer with just 5 bytes of memory. We can view the memory for this machine, representing empty memory locations as O and full memory locations using the letter representing the variable stored in that location. Initially, memory looks like OOOOO.
Let's allocate a 2 byte variable A. After creating that variable our computer's memory looks like AAOOO. We have 3 bytes available, and a 3 byte variable can fit in those three contiguous bytes. Next, let's create a 1 byte variable B. We have a contiguous block of memory 1 byte long; in fact there are three possible locations for B. Let's put it right next to A, to try to save space. Our memory is now AABOO. We still have 2 bytes free, and they're contiguous, so the largest array we could create is 2 bytes.
And now the plot thickens: let's clear the variable A from memory. Clearing A from memory causes our computer's memory to look like OOBOO. The MEMORY function would indicate that there are 4 bytes free (4 O's in our memory view) but if we tried to use all of that memory, we would receive an out of memory error. The largest array we can create is 2 bytes long.
  2 个评论
Jan
Jan 2015-9-26
Very nice explanation, thank you.
I thought "Maximum possible array" as produced by memory() indicated exactly that: the contiguous available memory.
For total available memory I look at the line "memory available for all arrays"
Because maximum possible array is 300 MB this is probably too little for the temporary array that is created when executing the array. About 8*90 MB is 720 would be needed. Only 300 is contiguously available, thus an out of memory error is thrown.
What puzzles me is why I can't make it work using integer types (by casting shift to uint8)
Steven Lord
Steven Lord 2015-9-26
x = zeros(1, 100); x = uint8(x) would create a 1-by-100 double precision array x then create a 1-by-100 uint8 array and copy the elements from the double array to the uint8 array.
Something like zeros(1, 100, 'uint8') avoids creating the 1-by-100 double precision array first.
So how exactly are you casting/creating shift as a uint8 array?

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Matrix Indexing 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by