Comparing Duration Arrays is time consuming - How to improve my script?

Question

Robin Schäfer 2019-9-4

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/478879-comparing-duration-arrays-is-time-consuming-how-to-improve-my-script

回答： Steven Lord 2019-9-12

EDIT 12.9.2019: Due to the first comments i tried to explain my problem more in detail. I will try your suggestions and response in a comment EDIT END

Hello alltogether

i got a problem on my script while comparing duration arrays which is taking a huge amount of time .

What I want to do with my Script: dataG is read form .xls-file and contains time information on start and end times ( format 2 x 1 : 03-Aug-2019 10:06:12). data is read from .csv-file and contains two time vectors (format 1 x n : 9:31:52.000 ...). The time vector from data fits into the time vector from dataG. I want to cut the array from data corresponding to the given start

Here some excerpts from my script:

dataG=readtable(X{posG});            % data with start and end time
...
data=readtable(N{1});                 % continous time vector
...
    
tdur=table2array(dataG(posPG,7:8));     % is read as datetime format
[y, m, d]= ymd(tdur);
tdur=tdur-datetime(y,m,d,0,0,0);        % conversion to duration format
clock=table2array(data(:,2));           % is read as duration format
try    
    for post1=1:length(clock)            % Comparing durations - Here is where about 90% of the time is consumed
    if tdur(1)<clock(post1)
        break
    end
    end
    for post2=post1:length(clock)   
    if tdur(2)<clock(post2) || isnan(clock(post2 + 1))
        break
    end
    end
catch
end
data(post2:end,:)=[];           %cut data
data(1:post1,:)=[];

Matlab recognizes my variables automatically with 'readtable'.' tp' contains start time and end time.

In order to compare both variables I chose the conversion to duration arrays for both variables. For certain reasons, it is slowing down my whole script which are about 300 more lines and many for-loops and if-conditions. The function is called about 800 times, so time does matter..

The Profiler says, table2array is the most called function, 80-90 % of time on the first for-loop. I am pleased for any sugestion, to make my script faster! Thank you in advance =)

I think

Kind Regards

Rob

3 个评论
显示 1更早的评论隐藏 1更早的评论

Steven Lord 2019-9-4

If you describe at a higher level what you're trying to do with this code, we may be able to offer some guidance to help improve your code to accomplish your goal. Right now you've shown us a low-level view that makes me suspect you want to discretize your duration data, but I can't be certain.

Robin Schäfer 2019-9-12

@Steven Lord: You are completely right, i missed to tell you anything about what this script should do and what kind of data is available. I tried to make it more clear now!

@Star Strider: I've never used timetables before. I will give it a try! Thank you!

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Fabio Freschi 2019-9-4

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/478879-comparing-duration-arrays-is-time-consuming-how-to-improve-my-script#answer_390378

在 MATLAB Online 中打开

Not sure if I understand correctly the problem without data. I try:

idx = find(clock > tdur(1),1,'first')

2 个评论
显示无隐藏无

Robin Schäfer 2019-9-12

编辑：Robin Schäfer 2019-9-12

在 MATLAB Online 中打开

Incredibly! Looks much better than my approach and is exorbitantly faster (12 min for my whole dataset compared to >3h before)! Solved my problem!

A last question: I use now:

post1=find(clock>tdur(1),1,'first');
post2=find(clock>tdur(2),1,'first');

Is it possible to use the find-function in one line and get an array (e.g. post 1 x 2 ) as result? I get only the first value when I use tdur without (1):

post=find(clock>tdur,1,'first');

Fabio Freschi 2019-9-12

Not with find. We can try to make the exercise, but I am not sure if is worth the hassle

请先登录，再进行评论。

Answer 2

Steven Lord 2019-9-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/478879-comparing-duration-arrays-is-time-consuming-how-to-improve-my-script#answer_391610

So you want to extract only those subset of rows of your data that fall into a certain time span? If so, and if you choose to store your data in a timetable array, use a timerange as your row index.

Depending on what you want to do with the data from that smaller time span, some of the other timetable related functions may be of use (I'm thinking specifically of retime or some of the data preprocessing functions like the grouping functions.)