Mapreduce with tall array in matrix multiplication

Question

chen zhe 2024-6-26，14:54

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2132266-mapreduce-with-tall-array-in-matrix-multiplication

评论： chen zhe 2024-6-26，16:06

Hi,

I am processing huge dataset (billions rows), so I used tall array in my code. While I found the tall array are not support general matrix multiplication. The '*' only works in a limited conditions.

I am wondering if we can combine mapreduce to perform matrix multiplication with tall arrays ? Below is a example shows the error.

A = tall(ones(200,2));
B = tall(ones(2,3));
values = A*B; 
gather(values)

Error using tall/mtimes>iVerifyAtLeastOneScalar

Matrix multiplication of two tall arrays requires one of them to be scalar.

Learn more about errors encountered during GATHER.

Error in * (line 31)

[X,Y] = iVerifyAtLeastOneScalar(X,Y,"MATLAB:bigdata:array:MtimesBothTall");

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Torsten 2024-6-26，15:31

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2132266-mapreduce-with-tall-array-in-matrix-multiplication#answer_1477546

编辑：Torsten 2024-6-26，15:32

在 MATLAB Online 中打开

Would this work for you ?

B = ones(2,3);
A = tall(ones(200,2));
values = A*B;
gather(values)
Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: 0% complete
- Pass 1 of 1: Completed in 0.21 sec
Evaluation completed in 0.37 sec
ans = 200x3
     2     2     2
     2     2     2
     2     2     2
     2     2     2
     2     2     2
     2     2     2
     2     2     2
     2     2     2
     2     2     2
     2     2     2
<mw-icon class=""></mw-icon>
<mw-icon class=""></mw-icon>