Why does synchronize omit and duplicate rows from component timetables?

Question

Will S 2023-12-15

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2061182-why-does-synchronize-omit-and-duplicate-rows-from-component-timetables

评论： Will S 2023-12-16

timetables.mat

When I synchronize two timetables, the result omits or duplicates rows from the component timetables, depending on options. The attached timetables.mat file contains two timetables, tt and TT, that reproduce the problem. Both TT and tt have a sample rate of 5e6, and were built using array2timetable, where I specified the vector of values, the sample rate (5e6), and a start time.

load('timetables.mat', 'TT', 'tt')
missing_rows = synchronize(TT, tt, 'regular', 'fillwithmissing', 'SampleRate', 5e6);
duplicate_rows = synchronize(TT, tt);

In missing_rows, variable 4 ("tt"), rows 14588-14594 are filled with NaNs, but data exist at these times, as seen in timetable tt rows 488-494. These kinds of omissions can be seen in many other places if you run stackedplot(missing_rows).

In duplicate_rows, look at rows 14105-14119 variable 4 ("tt"). TT and tt share times, but in these rows of duplicate_rows, each time is listed twice with NaNs interspersed. Again, this occurs in many places.

Is there a way to synchronize these tables without the missing or duplicate row problems?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Will S 2023-12-16

Here are some screenshots of the rows that I describe.

tt

missing_rows

duplicate_rows

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Will S 2023-12-16

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2061182-why-does-synchronize-omit-and-duplicate-rows-from-component-timetables#answer_1372797

在 MATLAB Online 中打开

Since posting, I figured out that the cause of this behavior is floating point errors in the timetable times. I'm not sure why synchronize leaves out data in the missing_rows case, but the give-away for floating point errors comes from taking the difference between the rows that look like duplicates in the duplicate_rows case.

>> duplicate_rows.Time(14117) - duplicate_rows.Time(14116)
ans = 
  duration
   4.4409e-19 sec

I let matlab create the timetales with array2timetable, where I provide the sample rate and start time. Perhaps the floating point errors come from the different start times I provide?

There are already solutions posted to this floating point problem:

How to increase the tolerance for comparing time points when synchronizing timetables? - MATLAB Answers - MATLAB Central (mathworks.com)

I wanted to add an answer for my application, based on this related question.

Synchronize timetables with tolerance of 5 minutes - MATLAB Answers - MATLAB Central (mathworks.com)

In my case, I'm recording from data acquisition equipment with a constant sample rate, so I need to syncrhonize within the tolerance of the time step set by my sample rate. Therefore, prior to synchronizing timetables, I just round their times so the time steps have the same number of decimal places as 1/sample_rate:

tt.Time = seconds((round(seconds(tt.Time), -floor(log10(1 / sample_rate)))));