How to pre-allocate changeable size arrays in a for-loop?

11 次查看（过去 30 天）
Ana Castanheiro
Ana Castanheiro 2017-3-20
评论： Guillaume 2017-3-24
Hi all!
I have a quite large script, where I deal with different particle size classes (or intervals) depending on given input. Each of the size classes (i=1:NrInt) will result in a cell array with a certain size (nr rows depends on nr of particles, nr columns depends on nr of chemical elements present). These arrays are then submitted to a number of operations in order to produce correspondent pie charts (1 pie per particle size interval).
I received part of this script from a previous colleague, which I then tried to adapt to my specific problem. After lots of error and trial, the script seems to be working just fine, but I still need to improve speed. As several variables are changing size inside the loop, I'm getting the warning preallocate messages all over the place.
The code for that part is in attachment. Can someone help me out?
Many thanks!
  3 个评论
显示 1更早的评论
Ana Castanheiro
Ana Castanheiro 2017-3-20
Indeed, Adam. I profilled my script and that specific part does not seem to take a lot of time, at least compared to the total time. I have a few dialog boxes to request for input, and these are always more time consuming. Nevertheless, I'd like to solve the warning messages anyway.
Also, I think it might be something simple to do and easily repeatable for all the changeable size variables. So, I'm still looking for suggestions.
Adam
Adam 2017-3-20
编辑：Adam 2017-3-20
Ah, well, dialog boxes will skew profiler results probably in terms of percentage time taken in each part of the program. If the program is waiting for user input it will be sat there with the timer ticking away on that function, giving an erroneous evaluation of the time spent on the function itself.
Your code is too complicated for me to just glance at in the time I have though and make any valid suggestions.
You really ought to make use of blank lines in code for readability! I wall of text covering 30 lines or more is really hard to read.
I notice you are using cell arrays and these are never good for performance. Maybe they are totally necessary here, but if you can in any way use numeric arrays instead of cell arrays that would likely improve performance.
As for preallocation, sometimes you simply cannot do it if you have no idea how big your array will be beforehand. If you can estimate an upper bound on the size then you can presize it to this and then just trim it down to the smallest size it can be after the loop. Sometimes I do this if I can make a sensible estimate. Otherwise you can tell it to ignore that warning message. You are right to look into the message and try to solve it first though - only disable warnings when you have evaluated them and are happy to ignore them for valid reasons.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Guillaume
Guillaume 2017-3-23
编辑：Guillaume 2017-3-24
I'm with Adam and Dhruvesh, your code is very difficult to parse. Better indentation (select all code and press CTRL+I), more white spaces and comments would greatly help.
At a quick glance, I fail to see which variable cannot be pre-allocated. They all seem to be indexed by i which you know will have NrInt steps.
Like Adam, I wonder if all these cell arrays are necessary. There are also several number to string conversions. That's never going to be fast.
I also noticed several instances of
somevar = find(someexpression)
othervar(somevar) = ...
which can be replaced by
othervar(someexpression) = ...
There's no point in using find to convert the logical array returned by someexpression into explicit indices when you can use that logical array directly for indexing. The find call just slow things down.
Note that if you cannot preallocate (which is perfectly fine) you can get rid of the warning either by right clicking on the squigly line and selecting Suppress ... on ..., or adding %#ok<AGROW> at the end of the line.
  3 个评论
显示 1更早的评论
Adam
Adam 2017-3-24
But all those variables in the loop are still being indexed by i, so when you get to the end of the loop they should all be of size NrInt so you could presize them all to that.
Guillaume
Guillaume 2017-3-24
The code is indeed a lot easier to read.
That the size of the arrays stored in each cell of the cells array differs does not matter. The number of cells of the cell array is fixed at NrInt. Hence all your cell arrays and vectors could be pre-allocated. The only variable whose size is unknown at the beginning of the loop appears to be TotLabel. That one however could be created after the loop.
So any cell array and vector can be predeclared with, e.g.
ClassHistFinal = cell(1, NrInt);
Nrparticles = zeros(1, NrInt); %for vectors
As said, TotLabel can be calculated after the loop has ended with:
TotLabel = [Label{:}];
There are a lot of what looks like intermediary results that are stored in cell arrays. Is it really necessary? Do you really need to keep all the intermediary results ClassNor, ElementsAllZero, etc. If not get rid of the indexing.
I personally prefer building strings with fprintf or fprintf and a format string, rather than string concatenation and num2str. So I'd rather have:
fprintf('Number of particles between %g and %g µm : %d(%g%%)\n', ThresholdVal(i:i+1), NrParticles(i), NrParticles(i)/TotalNrParticles*100)
than your
disp(horzcat(...
That won't have an impact on speed. I just find the former more readable and it's easier to customise the number display

请先登录，再进行评论。

更多回答（1 个）

Dhruvesh Patel
Dhruvesh Patel 2017-3-23
Your code indeed is difficult to read. However here are some general pointers which might help you undersatnd what is going on under the hood when MATLAB resizes an array. The way MATLAB works while resizing an array when more elements are asked for by the for-loop is nicely explained in the following answer. It talks about both, the normal arrays as well as cell arrays.
So, it is always a good idea to take an estimate for the size and pre-allocate using that as this would mean that MATLAB will not have to resize atleast till the size reaches this estimated value. This would improve execution time as well as reduce memory fragmentation. Ideally if you have an upper bound for your loop iterations (looks like its 'NrInt' in your case) you can pre-allocate using that.

类别

在 Help Center 和 File Exchange 中查找有关 Whos 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by