Fasten the speed of assigning values to a large tensor
3 次查看(过去 30 天)
显示 更早的评论
Hi! Now I have a for-loop assigning values to some entries of a large tensor at every iteration. But it is too slow since the tensor is large. Now I am thinking if there is any way to fasten this speed. One simple example is shown below. Here I used 'ndSparse' function to create a four dimensional tensor instead of 'zeros' function. The link of 'ndSparse' is https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays. ndSparse generalizes 'sparse' function from sparse matrix to high dimensional sparse tensor.
% Simple example
M = 500;
A = ndSparse.build([M,M,M,M]);
n = 10;
for i=1:n
disp(i)
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
Is there any way to make it faster?
Thank you very much!
回答(2 个)
Jan
2021-4-28
编辑:Jan
2021-4-28
Avoid creating index vectors explicitly:
idx1 = a1:b1;
A(idx1, :) = tmp; % Slow
A(a1:b1, :) = tmp; % Faster
In the first case, each element of idx1 is tested, if it is a valid index:
% For each index something like this is performed:
if idx(1) == flor(idx(1)) && idx(1) > 0 && idx(1) < size(A, 1)
With a1:b1 Matlab checks the validity of the first and last element only.
Please post a minimal working example. Before the readers can test, if their suggestion is faster, they currently have to implement your code based on guesses, what
A = create a sparse all zero 6-D tensor of size (M,M,M,M,M,M);
or
idx1 = a1:b1; (b1-a1=m)
explicitly means.
2 个评论
Jan
2021-5-3
@Jason Yang: You are right, ndSparse is infact an exception, because this user defined class is not supoorted by Matlab's JIT acceleration.
Matt J
2021-5-3
编辑:Matt J
2021-5-3
You can save some time if you pre-allocate the array:
% Simple example
M = 400;
n = 10;
tic
A = ndSparse.spalloc([M,M,M,M],(M/n)^4*n); %Pre-allocate
%A=ndSparse.build([M,M,M,M]);
for i=1:n
idx1 = M/n*(i-1)+1:M/n*i;
idx2 = M/n*(i-1)+1:M/n*i;
idx3 = M/n*(i-1)+1:M/n*i;
idx4 = M/n*(i-1)+1:M/n*i;
tmp = randn([M/n,M/n,M/n,M/n]);
A(idx1,idx2,idx3,idx4) = tmp;
end
toc%Elapsed time is 1.282638 seconds.
I don't think there are great gains to be made, however. Most of the computation is being spent just generating the entries for the array:
clear B
tic;
[B{1:4}]=ndgrid(1:M/n);
B = reshape( cat(5,B{:}) ,[],4);
[C,vals]=deal(cell(n,1));
for i=1:n
C{i}=B+M/n*(i-1);
vals{i} = randn(prod([M/n,M/n,M/n,M/n]),1);
end
C=cell2mat(C); vals=cell2mat(vals);
I=sub2ind([M,M,M,M],C(:,1),C(:,2),C(:,3));
J=C(:,4);
toc; %Elapsed time is 1.499094 seconds.
tic
S=sparse(I,J,vals, M^3,M, (M/n)^4*n);
toc; %Elapsed time is 0.344042 seconds.
tic;
A=ndSparse(S,[M,M,M,M]);
toc;%Elapsed time is 0.016531 seconds.
0 个评论
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!