Dealing with large timetables

6 次查看（过去 30 天）

Lutetium 2020-10-28

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/629233-dealing-with-large-timetables

评论： Star Strider 2020-10-29

I generated two timetable [283580428x5] and [21317257x2] and have also written them to a file using writetimetable().

Now I would like to synchronize them using synchronize [table1, table 2] but I get 'out of memory' very soon after executing the command.

Is there a way to process these table blockwise or any advice how I can handle these kind of large timetable efficiently w/o running out of memory? I appreciate any help or guidance

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Star Strider 2020-10-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/629233-dealing-with-large-timetables#answer_527028

The only option I can suggest to you is in the timetable documentation section on Extended Capabilities under Tall Arrays. (I have no personal experience with huge data sets.)

4 个评论
显示 2更早的评论隐藏 2更早的评论

Lutetium 2020-10-29

在 MATLAB Online 中打开

Thanks, I got it to work now with your input. It may be the lack of experience but I agree, figuring out the correct import options can be quite frustrating.

In the end, I converted to a tall array and performed the synchronization. With write the synchronization is executed.

Using the datastore, I can recall the timetable again without any importing options. In temp, the data is stored a *.mat files.

ds1 = tall(variable)
ds2 = tall(variable)
ds3 = synchronize(ds1,ds2)
location = 'E:\Temp'
write(location,ds3)
location = 'E:\Temp'
ds = datastore(location)