Matching two arrays that have similar values

2 次查看（过去 30 天）
HRmatlab
HRmatlab 2014-7-18
编辑： Arun Mathew Iype 2014-7-24
I have two folders that have time stamped files. The time stamps are in this format: MMDDYY HHMMSS.### where ### is the number of milliseconds.
I need to compare the contents (file names) of the two folders and generate a new array (or structure) that contains all the files from both folders that have time stamps within 12 milliseconds of each other.
I have manipulated the file name strings and generated numeric arrays with the file stamps but then I still have to compare two large numeric arrays to each other with double loops etc.
That just takes up too much time (I have dozens of folders each containing thousands of files).
I was wondering if there is a quicker easier way to do it?
Thank you in advance
  2 个评论
dpb
dpb 2014-7-18
Not sure it would be any faster in the end, but a sorted combined vector of all would allow a single processing pass from the beginning to pick the ones sequentially that are no farther than M millisec from present. Then increment to the next beyond that and repeat.
Cedric
Cedric 2014-7-18
编辑：Cedric 2014-7-18
Sort first as proposed by dpb, and then you can cluster based on time difference (using DIFF and e.g. ACCUMARRAY).

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Arun Mathew Iype
Arun Mathew Iype 2014-7-24
编辑：Arun Mathew Iype 2014-7-24
It seems sorting the data first and then using some matching algorithm is faster by around 65 times on my system.
What I did was to sort the two numeric arrays using “sortrows”. Then I match each entry in the first array with entries which fall in 12 millisec duration in the second array. Since the array is sorted, the search can be stopped if you get a non-match after a match. Additionally, the next element in the first array may start the search from the first matched element in second array for the previous entry in the first array.
The following is the test script based on an input of around 1000 rows in each file array. >> array_grouping Time for generating matches using simple for loops =5.2695 Time for generating matches using sorting =0.082277
Image : Input files
Image : Output matching

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by