Efficient summing of parts of an array

4 次查看（过去 30 天）
Ife
Ife 2023-7-24
评论： Ife 2023-7-25
I have a 4D dataset that I'm expressing as a sparse 2D array of 1s and 0s. The 4D data has dimensions NPixHeight by NPixWidth by NBins by NFrames, and as a sparse array has the dimensions NPixHeight*NFrames by NPixWidth*NBins.
As part of my pre processing of I want to compress the 2D array by summing along the F dimension in groups, the size of the group is defined by nExps in the below code. To realise this I create and transform matrix T that after multiplication with the original array outputs a NPixHeight*nExps by NPixWidth*NBins array.
This last step is pretty slow for our planned application and I was hoping someone might be able to point me in the right direction to speed it up.
Here's an excerpt from the code
%% Data Reshape
% number of exposures for macro time
nExps = NFrames/macroFrames;
% generates sparse array of counts Ypix*NFrames by Xpix*Bins where
% NFrames is limited by macrotime
yNew = y+(f-1).*NPixHeight;
xNew = x+(b-1).*NPixWidth;
SpCounts = sparse(yNew,xNew,1,NPixHeight*NFrames,NPixWidth*NBins); % SLOW
% transform matrix that sums a number frames set by macro time
T = repmat(speye(NPixHeight,NPixHeight),1,macroFrames);
TCell = repmat({T}, 1, nExps);
clearvars T
T = blkdiag(TCell{:});
% multiplication and reshaping to output
% 4D array of Y x X x Bins x NumberMacroExposures
camData.counts = sparse(size(SpCounts,2),size(T,1));
camData.counts = full(T*SpCounts).'; % SLOW
camData.counts = reshape(camData.counts,NPixWidth*NBins,NPixHeight,nExps);
camData.counts = reshape(pagetranspose(camData.counts),NPixHeight,NPixWidth,NBins,nExps);
(I know the double reshape is awkard but its not enough of a bottleneck to bother fixing)
  3 个评论
显示 1更早的评论
Ife
Ife 2023-7-24
编辑：Ife 2023-7-24
It depends on the user input but for a "worst case" with:
NPixHeight = 192
NPixWidth = 126
NBins = 344
NFrames = 247360
macroFrames = 30920
nExps = 8
gives a sparse array of 47493120 x 43344 with a sparsity of 6.2715e-05.
Forgive the stupid question but what is a mex routine...?
the cyclist
the cyclist 2023-7-24
编辑：the cyclist 2023-7-24
A MEX file is a function that calls C,C++, or Fortran from within MATLAB.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Matt J
Matt J 2023-7-24
I would recomend that you maintain the 4D array in ndSparse form instead,
https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays
at which point you can do,
temp = reshape(yourArray,[],NFrames)*kron(speye(nExps),ones(macroFrames,1));
result=reshape(temp,NPixHeight, NPixWidth , NBins, []);
  1 个评论
Ife
Ife 2023-7-25
Thanks! Due to what I'm using this code for I can skip the 4D sparse representation then reshape steps and readout my data directly into the correct size/orientation. My code is running 2x - 5x faster depending on the size of the datasets now!

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by