Create/deal big binary sparse matrices
显示 更早的评论
Hi, I'm dealing with really big binary sparse matrices and I need to manipulate them (i.e allocate memory, multiplication etc.) I'm aware of the function sparse and I use it in my code. the first step is taking 4.5 second with parameters (t=8000,n0=2000).
And with bigger matrices, it takes about 2min whereas the rest of the code is taking about 5secs...
The question how one can efficiently allocate/create (a big) random binary sparse matrix?
tic
%%step 1 create random matrix with proba p=0.05
%I allocated first with sparse(t,n0) but the result was the same
%tried also false(t,n0)
A=rand(t,n0)<p;
toc
%step 3
tic
%finding number of rows of A that have 1 at both column i and column j
%by multiplying it with its transpose
B=sparse(A)'*sparse(A);
%getting numbers (i.e counts)
W=triu(B,1);
edges=(W>=meanvalue);
toc
Thanks in advance for your time and help.
1 个评论
Bruno Luong
2011-2-28
It is not clear to me what take time. One thing for sure: don't use SPARSE as you did: i.e., generate full matrix then convert with sparse command. The efficient SPARSE command is with the form
SPARSE(rows, cols, values, ...). DO GENERATE SPARSE from the start.
回答（3 个）
Walter Roberson
2011-2-28
0 个投票
I doubt it is the memory allocation or creation of the sparse matrix that is taking the time. I would think it much more likely that it is the matrix multiplication that is taking the time, as that will result in a matrix which is less sparse than the original matrix.
4 个评论
Mehmet Candemir
2011-2-28
Walter Roberson
2011-2-28
编辑：Matt J
2020-10-20
Seeing as random values are independent of each other, build in pieces.
NP = 10; %number of fragments to piece together
PSize = n0/NP; %fragment width
rowcol = cell(K,2);
for K = 0 : NP - 1
[row,col] = find(rand(t,PSize) < p);
rowcol{K,1} = row;
rowcol{K,2} = col + Psize * K;
end
row = vertcat(rowcol{:,1});
col = vertcat(rowcol{:,2});
A = sparse(row,col,1);
Mehmet Candemir
2011-2-28
Walter Roberson
2011-2-28
Sorry, the line should be
rowcol = cell(NP,2);
Bruno Luong
2011-2-28
See my comment above.
Instead of
A=rand(t,n0)<p;
Use sparse directly
A = logical(sprand(t, n0, p)); % OR
A = spones(sprand(t, n0, p));
Bruno
3 个评论
Mehmet Candemir
2011-2-28
Bruno Luong
2011-2-28
Your timing does not mean much:
1) the full matrix cannot even be use for large dimension
2) the time needed later to convert to sparse is not taken into account.
Mehmet Candemir
2011-2-28
Bruno Luong
2011-2-28
m = 80000;
n = 10000;
p = 0.001;
nel = m*n*p;
rows = ceil(m*rand(1,nel));
cols = ceil(n*rand(1,nel));
A = sparse(rows, cols, 1);
B = A'*A;
...
类别
在 帮助中心 和 File Exchange 中查找有关 Matrix Indexing 的更多信息
产品
2011-2-28
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
