Advice on Speeding up Loops
2 次查看(过去 30 天)
显示 更早的评论
I need to run the code below with the parameter amnt equal to about 10^8. The loops are very expensive and I have written in a way to time the execution as a function of the parameter, amnt. The figure shows the execution time as a function of the matrix sizes (amnt) and was run with 12 helpers (cores). Via regression it seems when amnt=10^8 it will take over a month to run.
In a nutshell, the purpose of this code is to bin 4 matrices and then find the most common bin for each.
I am asking:
- Can anyone recommend a way to vectorize the loop(s) or another way to speed it up?
- The use of LocB in the ismember function is being flagged by Matlab as being inefficient when using a parfor loop. I don't understand how to make better use of it, i.e. fix it. Any suggestions?
Thanks!
% Test problem for troubleshooting
rang = 100;
sz = 10^8;
% Some random matrices for this test case.
mat1t = randi(rang,[sz 1]) + 50*rand([sz 1]);
mat2t = randi(rang,[sz 1]) + 40*rand([sz 1]);
mat3t = randi(rang,[sz 1]) + 30*rand([sz 1]);
mat4t = randi(rang,[sz 1]) + 20*rand([sz 1]);
times = [0,0,0,0,0,0,0,0];
for tm1 = 1:length(times) % Loop just used for timing purposes.
amnt = 5^tm1; % Ultimately I want this to be 10^8, i.e. the entire matrices.
%amnt = numel(mat1(:)); % ALL
mat1 = mat1t(1:amnt); mat2 = mat2t(1:amnt); mat3 = mat3t(1:amnt); mat4 = mat4t(1:amnt);
NP = 50;
dl = max([range(mat1),range(mat2),range(mat3),range(mat4)])/NP;
tol = dl/2;
low = min([min(mat1),min(mat2),min(mat3),min(mat4)]);
high = max([max(mat1),max(mat2),max(mat3),max(mat4)]);
gridp = [low:dl:high]';
datap = [mat1 mat2 mat3 mat4];
whatBin = zeros(size(datap));
tic
for k = 1:4 % Loop over the 4 matrices
gridp1 = [gridp zeros(size(gridp))];
datap1 = [datap(:,k) zeros(size(datap(:,k)))];
tolp1 = [tol Inf];
[LIA,LocB] = ismembertol(gridp1,datap1,1,'ByRows',true,'OutputAllIndices',true,'DataScale', tolp1);
parfor kj = 1:length(mat1(:)) % Loop over all matrix elements
for kkj = 1:numel(LIA) % Loop over all bins
if ismember(kj,LocB{kkj}) % The use of LocB here is flagged by Matlab as being inefficient. How to fix?
whatBin(kj,k) = kkj;
break
end
end
end
end % End of loop over all 4 matrices
% Find the unique rows of the whatBin matrix
[u,I,J] = unique(whatBin, 'rows', 'first');
npix = size(u,1);
paretoPix = zeros([npix 1]); % Number of pixel occurances for a vector of metrics.
parfor j = 1:npix
paretoPix(j) = nnz(all(whatBin == u(j,:),2));
end
Results = [u paretoPix]; Results = sortrows(Results,-1*(size(whatBin,2)+1));
times(tm1) = toc;
end % end of times loop
semilogx(5.^(1:length(times)),times,'-o')
xlabel('Matrix Elements')
ylabel('Time (s)')
In this figure, the matrix size is the parameter amnt. Using these values in the curve fit app, the regression equation (2nd order polynomial) indicates that when amnt=10^8 it will take ~month!
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/747064/image.png)
4 个评论
Rik
2021-9-23
I'm on mobile, so your image is not much help.
Can't you use the third output of histcounts? That would tell you which persons are in which bin.
回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!