readtimetable from a java stream instead of file

2 次查看（过去 30 天）
Marc Vaillant
Marc Vaillant 2020-2-18
评论： Marc Vaillant 2020-4-24
TL;DR
Why the heck doesn't matlab provide overloads for IO functions, e.g. readtimetable, that take a low level stream? In particular, is there a way to have readtimetable read from a lowel level file stream instead of a file?
Details
Timetables and tables are nice datastructures but there is a significant roadblock to their interoperability with other languages. Their binary serialized form in .mat files is proprietary, so no one knows how to read/write them from other languages. You can read/write them as ascii--using say csv as a data exchange format--via readtimetable/writetimetable, but then their size blows up by a factor of 3 or 4. When you have timetables that would be 500 MB as mat files, and all of a sudden they become 2gb, that's a problem. You can compress the csvs, but alas, there's no real way to read compressed files on the fly using standard parsing functions like readtimetable. There's this which consoildates the information out there on trying read zip files on the fly without uncompressing and writing to a file first, which basicaly gets you a java file stream that can read from a zip file directly, but you can't pass it to readtimetable. The only option would be to spin your own csv to timetable parser that operates on a java file stream, but that seems silly given that the functionality is already there in readtimetable. Is there another option?
  2 个评论
Peter Perkins
Peter Perkins 2020-4-14
Marc, you are correct, and it's something that we are looking at. Have you tried using parquet? I confess that I am not really up on parquet, but my understanding is that lots of software can create that format, and in recent versions (looks like R2019a) there are readparquet and friends.
Marc Vaillant
Marc Vaillant 2020-4-24
Thanks Peter. I haven't tried using parquet, but that seems like a reasonable compact data exchange format. I will take a look.
Thanks again,
Marc

请先登录，再进行评论。

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Tables 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by