Create/deal big binary sparse matrices

14 次查看（过去 30 天）
Mehmet Candemir
Mehmet Candemir 2011-2-28
编辑： Matt J 2020-10-20
Hi, I'm dealing with really big binary sparse matrices and I need to manipulate them (i.e allocate memory, multiplication etc.) I'm aware of the function sparse and I use it in my code. the first step is taking 4.5 second with parameters (t=8000,n0=2000).
And with bigger matrices, it takes about 2min whereas the rest of the code is taking about 5secs...
The question how one can efficiently allocate/create (a big) random binary sparse matrix?
tic
%%step 1 create random matrix with proba p=0.05
%I allocated first with sparse(t,n0) but the result was the same
%tried also false(t,n0)
A=rand(t,n0)<p;
toc
%step 3
tic
%finding number of rows of A that have 1 at both column i and column j
%by multiplying it with its transpose
B=sparse(A)'*sparse(A);
%getting numbers (i.e counts)
W=triu(B,1);
edges=(W>=meanvalue);
toc
Thanks in advance for your time and help.
  1 个评论
Bruno Luong
Bruno Luong 2011-2-28
It is not clear to me what take time. One thing for sure: don't use SPARSE as you did: i.e., generate full matrix then convert with sparse command. The efficient SPARSE command is with the form
SPARSE(rows, cols, values, ...). DO GENERATE SPARSE from the start.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（3 个）

Walter Roberson
Walter Roberson 2011-2-28
I doubt it is the memory allocation or creation of the sparse matrix that is taking the time. I would think it much more likely that it is the matrix multiplication that is taking the time, as that will result in a matrix which is less sparse than the original matrix.
  4 个评论
显示 2更早的评论
Mehmet Candemir
Mehmet Candemir 2011-2-28
I tried to use it but I think that there may be a problem in your code you used twice K.
rowcol = cell(K,2);
for K = 0 : NP - 1
what is the first K?
Walter Roberson
Walter Roberson 2011-2-28
Sorry, the line should be
rowcol = cell(NP,2);

请先登录，再进行评论。

Bruno Luong
Bruno Luong 2011-2-28
See my comment above.
Instead of
A=rand(t,n0)<p;
Use sparse directly
A = logical(sprand(t, n0, p)); % OR
A = spones(sprand(t, n0, p));
Bruno
  3 个评论
显示 1更早的评论
Bruno Luong
Bruno Luong 2011-2-28
Your timing does not mean much:
1) the full matrix cannot even be use for large dimension
2) the time needed later to convert to sparse is not taken into account.
Mehmet Candemir
Mehmet Candemir 2011-2-28
correcting values
Elapsed time is 2.790257 seconds.
Elapsed time is 0.982951 seconds.
Elapsed time is 0.944833 seconds.

请先登录，再进行评论。

Bruno Luong
Bruno Luong 2011-2-28
m = 80000;
n = 10000;
p = 0.001;
nel = m*n*p;
rows = ceil(m*rand(1,nel));
cols = ceil(n*rand(1,nel));
A = sparse(rows, cols, 1);
B = A'*A;
...

类别

在 Help Center 和 File Exchange 中查找有关 Matrix Indexing 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by