Dealing with large timetables

5 次查看（过去 30 天）
Lutetium
Lutetium 2020-10-28
评论： Star Strider 2020-10-29
I generated two timetable [283580428x5] and [21317257x2] and have also written them to a file using writetimetable().
Now I would like to synchronize them using synchronize [table1, table 2] but I get 'out of memory' very soon after executing the command.
Is there a way to process these table blockwise or any advice how I can handle these kind of large timetable efficiently w/o running out of memory? I appreciate any help or guidance

请先登录，再回答此问题。

采纳的回答

Star Strider
Star Strider 2020-10-28
The only option I can suggest to you is in the timetable documentation section on Extended Capabilities under Tall Arrays. (I have no personal experience with huge data sets.)
  4 个评论
显示 2更早的评论
Lutetium
Lutetium 2020-10-29
Thanks, I got it to work now with your input. It may be the lack of experience but I agree, figuring out the correct import options can be quite frustrating.
In the end, I converted to a tall array and performed the synchronization. With write the synchronization is executed.
Using the datastore, I can recall the timetable again without any importing options. In temp, the data is stored a *.mat files.
ds1 = tall(variable)
ds2 = tall(variable)
ds3 = synchronize(ds1,ds2)
location = 'E:\Temp'
write(location,ds3)
location = 'E:\Temp'
ds = datastore(location)
Star Strider
Star Strider 2020-10-29
As always, my pleasure!
I have obnly used the datastore function once, and forgot to mention it as an option.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Tall Arrays 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by