very large matrix manipulation

3 次查看（过去 30 天）
Alex
Alex 2011-11-16
Hi I have a matrix that could end up being possibly 25x25 large. I need to do large manipulations on this matrix, for instance I need to generate a matrix of zeros B = zeros(n^m,m); where n and m are the dimensions of the original matrix. This obviously gives me an error because there is not enough computing power to do this. If I understand it right, any value in the matrix is a byte so that would be 25^25 bytes right? thats about 8*10^24 gb?! Whatever - Its too big. Can I use multiple cores to deal with the issue? Please can someone suggest how to get around this

请先登录，再回答此问题。

回答（2 个）

Walter Roberson
Walter Roberson 2011-11-16
In B = zeros(n^m,m) each value in the matrix will be a double precision number, which is 8 bytes.
You can use B = zeros(n^m,m,'uint8') to get one byte per entry.
Remember too that you have n^m by m, so that would be 25^25 * 25 = 25^26 = 2220446049250313080847263336181640625 bytes.
Using multiple cores is not going to help without a fundamental reorganization of your problem. 25^26 bytes requires a 121 bit address space, but the maximum implemented for any x86 type chips (the only thing MATLAB runs on these days) is 48 bits of address space. Even if you managed to partition the problem up in to 48 bits of address space per instance, you would need approximately 2^(121-48) = 2^73 = 9444732965739290427392 systems (that is close to 1E22)
You need a redesign.
  1 个评论
Alex
Alex 2011-11-16
Thanks Walter. If I show you the code, could you suggest an alternative way of doing it. I have a matrix of data, and I need to do a calculation on every combination of the data in the matrix - My analogy is like a number padlock - if you go through every combination you would eventually get the lock open. I am wanting to sum the inverses of the numbers within each combination, and keep the one that is the smallest
[code]
[m,n] = size(entry);
AT = entry.';
B = zeros(n^m,m);
B(1:n,m) = AT(:,m);%B is matrix of all combinations
p = n;
for k = m-1:-1:1
[I,J] = meshgrid(1:n,1:p);
p = p*n;
B(1:p,k:m) = [AT(I,k),B(J,k+1:m)];
end
value = zeros(length(B(:,1)),1);
for i=1:length(B(:,1))
for j = 1:length(B(1,:))
value(i) = value(i) + B(i,j)^(-1);
end
end
[c,ind] = min(value(:))
[/code]
I want c and the values that were used to make the combination
Thanks

请先登录，再进行评论。

Jill Reese
Jill Reese 2011-11-28
If you have access to Parallel Computing Toolbox and MATLAB Distributed Computing Server, you may find that distributed arrays will help you to work around the memory limitation. That is exactly what they are intended for.
  1 个评论
Alex
Alex 2011-11-28
Great thanks Jill, I was actually reading about Parallel Computing, but I couldnt work out how to apply it to my problem - have you got any examples, or know of any?

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by