How can I minimise the memory usage of the cat function

Question

James Anderson 2013-5-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/76535-how-can-i-minimise-the-memory-usage-of-the-cat-function

I have two very large data sets that I would like to concatenate using the CAT function. For example

    A = rand(10000,10000);
    B = rand(10000,10000);
    C = cat(3,A,B);
    Clear A B

My issue is around memory, the code above requires double the memory. Does anyone know a way that I can concatenate without using more memory? My inclination is to look at copy on write, and handle classes, but it is not clear to me if this would work? Thanks!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Iain 2013-5-21

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/76535-how-can-i-minimise-the-memory-usage-of-the-cat-function#answer_86200

Why not just create C to start with?

C = rand(10000,10000,2); ?

NB, C requires 1.4Gb of RAM. I suggest you avoid such big variables unless you have good reason.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Iain 2013-5-22

You could also add A to B, thus:

B(:,:,2) = A;

请先登录，再进行评论。

Answer 2

James Anderson 2013-5-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/76535-how-can-i-minimise-the-memory-usage-of-the-cat-function#answer_86257

Hi Iain, thanks for your suggestion. However I have shown a simplified example for a very complicated piece of code. It does not appear at the moment that I can create the concatenated matrix from the start. This is mainly because I would have to pre-allocate the data, and then my memory headroom would be limited for the interim calculations.