accelerate filter loop
显示 更早的评论
Hi everybody, I need to filter large data sets of about a million lines and some 15 columns. The data are echosounder readings, one data line per second. The simple thing I have to do is to remove those lines that exceed 3 m depth from one line to the next as these jumps are not plausible (they might be caused by fishes or air bubbles under the ship). I reduced my code to only find out the 'good' line numbers for later processing (instead of filtering the whole data set). Although this is quite simple, it takes ages and the speed seems to decrease being quick for the first 200 k lines but hen significantly slowing down. Does anybody know a quicker method to do such filtering? This is the code I use:
j=0;
for i=2:length(depthAlpha)
if abs(depthAlpha(i-1)-depthAlpha(i))<3
j=j+1;
goodlines(j,1)=i;
end
end
采纳的回答
Andrei Bobrov
2011-8-25
goodlines = find(abs(diff(depthAlpha(:)))<3)
更多回答（2 个）
Jan
2011-8-25
It is getting slower for more than 200'000 lines? This sounds like a missed pre-allocation. Although I'd prefer Andrei's solution, here is the pre-allocation for educational reasons:
goodlines = zeros(1, length(depthAlpha));
j = 0;
for i = 2:length(depthAlpha)
if abs(depthAlpha(i-1)-depthAlpha(i)) < 3
j = j+1;
goodlines(j) = i;
end
end
You can use TIC/TOC to compare the speed.
Christian
2011-8-26
0 个投票
2 个评论
Jan
2011-8-26
Could you please post a speed comparison? I'm collecting arguments for my "For-loops versus vectorization" investigation.
Andrei Bobrov
2011-8-26
Hi Jan! I look forward to publishing your investigation "For-loops versus vectorization"!
类别
在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
