have a 60027x3397 matrix. how to multiply each column with every other colum of same matrix (using mapreduce) as MATLAB runs out of memory
1 次查看(过去 30 天)
显示 更早的评论
Hello, I have a matrix A=60027x3397. I need to multiply each of the 3397 column with each other into a new matrix B. The combinations nC2 for 3397 turns out to be 5678106 and hence the MATLAB runs out of memory. I want to use MapReduce for this and was hoping to get an initial push for the same. I already have another matrix "k" 5678106x2 with the column indexes for the A for multiplication.
7 个评论
Walter Roberson
2016-5-6
How large are you expecting the adjacency matrix to be? And can it possibly be sparse? Will most values be non-zero or zero?
If you are taking the column sum, then use dot()
C = size(A,2);
numout = C * (C-1) / 2;
adj = zeros(numout, 1);
idx = 0;
for C1 = 1 : C - 1
AC1 = A(:,C1);
for C2 = C1+1 : C
idx = idx + 1
adj(idx) = dot(AC1, A(:,C2));
end
end
This if you are using the dot product directly as the weight. If you are filtering based upon value and that can be done by looking at individual results, then it would be better to write into a sparse matrix, in a different order than what I use here.
回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 MapReduce 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!