To scan a 1k dimensional matrix, parallel for is faster than simple for loop?

1 次查看(过去 30 天)
I need to scan a very huge matrix 15k x 10 x 1k
do you think parrallel-for is faster than simple for in this case? in a smaller matrix tic-toc says they are more or less the same.
Is there any fast way to go through a huge matrix elment by element?
  2 个评论
Stephen23
Stephen23 2023-3-14
"I need to scan a very huge matrix 15k x 10 x 1k"
What does "scan" mean? Such an array requires around 1.2 GB, which is not so large these days.
"do you think parrallel-for is faster than simple for in this case?"
I would not expect that, given the extra overhead in setting up a parallel loop, transferrring the data, etc. Whether there is a benefit to using parallel execution depends on many factors:
"Is there any fast way to go through a huge matrix elment by element?"
That depends mostly on what is happening inside the loop, which you have not told us.
Zahra Yousefi Darani
编辑:Jan 2023-3-14
samp = 1000;
parfor S = 1 : samp
stims = SampStim(S,:);
ErrorResp = nan(size(PhaseSpaceGLOBALparams,1),numel(MotherTau));
SimResp = nan(nTrials-1,10);
for T = 1 : 10
SimResp(:,T) = GuassianCDF_Global(MotherGamma,MotherLambda,MotherSigma,MotherCrit(2:end,T,S)',stims(1:end-1));
end
%%
RESP = binornd(1,SimResp);
for T = 1 : 10
for i = 1 : n
ErrorResp(i,T) = CrossEntr(RESP(:,T)',GuassianCDF_Global(PhaseSpaceGLOBALparams(i,3), ...
PhaseSpaceGLOBALparams(i,4),PhaseSpaceGLOBALparams(i,2),PhaseSpaceGLOBALparams(i,1),stims(1:end-1)));
end
[minval minid] = min(ErrorResp(:,T));
FITTEDPARAMS(S,T) = PhaseSpaceGLOBALparams(minid,2);
end
end
The huge matrix that is used in this code is MotherCrit(2:end,T,S)

请先登录,再进行评论。

采纳的回答

Suraj
Suraj 2023-3-30
Hey Zahra
For a matrix of size 15k x 10 x 1k, using parallel computing with "parfor" can potentially speed up the iteration process by utilizing multiple CPU cores to perform the computation in parallel. However, the actual speedup depends on the specific operations being performed within the loop, the available resources, and the memory usage.
In general, if the operations within the loop involve heavy computations, such as matrix multiplications, then using parallel computing can provide significant speedup. On the other hand, if the operations are simple, such as accessing individual elements of the matrix, then the overhead of parallelization may outweigh the benefits. You can read more about this here: https://www.mathworks.com/help/parallel-computing/decide-when-to-use-parfor.html .
To further optimize the iteration process, you can consider vectorizing the loop or using built-in functions that operate on the entire matrix at once. Vectorization and built-in functions are generally faster than using loops, especially for large matrices.If you need to perform a more complex operation that cannot be easily vectorized, you can try to optimize the loop using techniques such as preallocating arrays, minimizing memory usage, and avoiding unnecessary operations.
Hope this information helps.
  2 个评论
Zahra Yousefi Darani
I tried both parforloop and vectorized calculation. Interesingly when there are two super huge matrices and we want to do arithmetic calculation on them, vectorization is slower than when we try that calulation row by row or column by column with parfor. The reason is that first method occupies a lot of memory and the whole PC slows down.

请先登录,再进行评论。

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Loops and Conditional Statements 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by