Remove an element from a vector, using most computationally efficient solution

Question

Matt C 2016-7-5

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/293807-remove-an-element-from-a-vector-using-most-computationally-efficient-solution

评论： Matt C 2016-7-5

Hello, I recently wrote a function that removes elements from common vectors. It is a fairly elegant and simple piece of code, and it operates great when dealing with vectors that contain tens of thousands of elements; however, when I execute this code on vectors with tens of millions of elements, my code takes an exponentially longer time to execute (~1 minute vs ~1 week). I am wondering if I am using the "best memory practice" or most efficient solution, or if you guys can recommend an alternate method. Also, a point of note is that I am using 90% of the memory available to my machine (8 GB).

Specifically,

currentLength = length(vector1)
while i ~= currentLength+1
      if vector1(i) == someCondition
          % Removing data from all common indices of vector1, vector2, vector3, etc, through the varargin command
          vector1(i) = [];
          for i2 = 1:1:numOfVariableInputs
              varargin{i2}(i) = [];
          end
          i = i - 1;
          currentLength = currentLength-1;
      end
      i = i + 1;
end

I might be wrong, but I believe that using the =[ ] element removal technique that I have mentioned is continuously creating and destroying temporary variables to reduce the indices of the vectors.

Any help or thoughts to improve my code execution would be greatly appreciated!

Cheers,

- Matt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Thorsten 2016-7-5

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/293807-remove-an-element-from-a-vector-using-most-computationally-efficient-solution#answer_227752

编辑：Thorsten 2016-7-5

在 MATLAB Online 中打开

If your "someCondition" is indepent of variable "i" in your original post, you can do it like this:

idx = vector1 == someCondition;
vector1(idx) = []; 
for i = 1:numOfVariableInputs
  varargin{i}(idx) = [];
end

3 个评论
显示 1更早的评论隐藏 1更早的评论

Adam 2016-7-5

在 MATLAB Online 中打开

idx = ismember( vector1, someCondition );

should work.

Matt C 2016-7-5

Wonderful - Thanks for the simplification, guys! My code now executes in about 30 seconds instead of the predicted 1-week. Gotta save the bits and bytes wherever we can!

请先登录，再进行评论。

Answer 2

Adam 2016-7-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/293807-remove-an-element-from-a-vector-using-most-computationally-efficient-solution#answer_227749

编辑：Adam 2016-7-5

在 MATLAB Online 中打开

Are you not able to create a logical vector pointing to the elements that should be removed and then remove them all at once at the end?

You haven't included details of exactly what conditions you are testing so it is difficult to judge, but usually you wouldn't want to be resizing a vector while you are looping around it. Even if you correctly adjust your indices to make sure you don't overflow it is just very inefficient and I am not surprised its performance is exponential.

If instead you simply take not of the indices that should be deleted then you just delete them in a single command after the loop. This does mean maintaining another vector of the same length which is an issue for memory, but can you not divide your vectors up into smaller chunks to deal with this?

e.g.

toDelete = false( size( vector1 ) );
for i = 1:numel( vector1 )
   if vector1(i) == someCondition
      toDelete(i) = true;
   end
end
vector1( toDelete ) = [];
vector2( toDelete ) = [];

etc.

Obviously if you can vectorise your condition rather than running a loop then it would be even faster, but that depends what the condition is you are testing against.