out of memory error

7 次查看(过去 30 天)
Mark Bodner
Mark Bodner 2022-9-28
评论: Jan 2022-9-29
I'm trying to join to large arrays (rawS=43914x11 and rawP=23772085x11). My code is
out1=cellfun(@num2str,rawS,'un',0)
out2=cellfun(@num2str,rawP,'un',0)
[idxP,idxS] = ismember(out2(:,1),out1(:,1));
out=rawS(idxS(idxP),[1,2,3,4,5,6,7,8,9,10,11]);
out(:,[12,13,14,15,16,17,18,19,20,21,22]) - rawP(idxP,[1,2,3,4,5,6,7,8,9,10,11]);
This ran just fine the first time I ran it, but runs out of memory now (seemingly at different points each time) on the second line for "out2" . No other programs or processes are running. I Tried restarting the computer of course, using tall arrays, deleting variables after they are not needed and so forth, but nothing seems to works. Short of going next to distributed arrays to parallel workers, I would appreciate greatly any ideas or suggestions. Thanks.
  5 个评论
Mark Bodner
Mark Bodner 2022-9-28
Sorry, I had accidently mistyped the code--the last line is actually
out(:,[12,13,14,15,16,17,18,19,20,21,22]) = rawP(idxP,[1,2,3,4,5,6,7,8,9,10,11]);
The installed RAm is 64 GB.
The code with the tall approach was simply:
t=tall(rawP);
out1=cellfun(@num2str,rawS,'un',0)
out2=cellfun(@num2str,t,'un',0)
[idxP,idxS] = ismember(out2(:,1),out1(:,1));
out=rawS(idxS(idxP),[1,2,3,4,5,6,7,8,9,10,11]);
out(:,[12,13,14,15,16,17,18,19,20,21,22]) = rawP(idxP,[1,2,3,4,5,6,7,8,9,10,11]);
Jan
Jan 2022-9-29
@Mark Bodner: Again, the code would be better to read, if you replace [12,13,14,15,16,17,18,19,20,21,22] by 12:22 .

请先登录,再进行评论。

采纳的回答

Walter Roberson
Walter Roberson 2022-9-28
Pre-allocate out to the final size. This will reduce temporary memory use. Also, using : is more memory efficient than indexing at the list of columns.
out = zeros(numel(idxP), 22);
out(:, 1:11) = rawS(idxS(idxP), :);
out(:, 12:24) = rawP(idxP, :);

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Startup and Shutdown 的更多信息

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by