Is it possible to save partial arrays without compression? matfile() forces compression during save.

6 次查看（过去 30 天）
Thomas Richner
Thomas Richner 2018-8-23
编辑： Thomas Richner 2018-8-24
Hi, I would like to incrementally write arrays to a .mat file without compression. I know how to save files without compression using save(...,'-nocompression'). That won't work for me, because I often don't have the RAM to hold the variable in memory before saving. I also know how to save files incrementally with matfile(), but that won't work here because it forces compression. I've tried tricking matfile() into not using compression by initializing the array with an uncompressed variable either with save() or with h5create(), but neither worked. Perhaps I could use h5write(), but it seems so low level--it doesn't support many data types such as logical arrays and char arrays. I could probably write my own class to do this, but it is a bigger job. Is there a better way to write partial arrays to .mat or is there a better file type all together? Thanks for your advice.
You might ask about my application and why I need this feature: I work with large files (50-500 GB) that do not benefit much from compression. With a modern nvme SSD, large portions of uncompressed data load in just a few seconds, but compressed data takes several minutes to load due to singlecore CPU bottlenecks. If multiple cores were used, then compression might be viable.

请先登录，再回答此问题。

采纳的回答

Jan
Jan 2018-8-24
What about using a binary file together with memmapfile?

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Data Import and Analysis 的更多信息

标签

产品


版本

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by