Handling large data sets

1 次查看（过去 30 天）
v k
v k 2019-5-23
编辑： per isakson 2019-5-24
I have a Simulink model which calculates three-dimensional data every simulation time step, resulting in a four-dimensional array. As it can get large, I use the "to-file" block to save the data in a time series (only timeseries supports multidimensional data). All that works fine.
Now I would like to plot the data using an .m file. However, I cannot load the complete timeseries into workspace as it is too large (multiple GB). How can I access only a part of the time sereis, e.g. all data for one specific time sample? I tried using loading parts of .MAT-files, but indexing is limited and I did not get it working. getsamples on the other hand needs the time series to be loaded into workspace.
Is there any solution or has anyone ideas how to access parts of time series data?

请先登录，再回答此问题。

回答（1 个）

per isakson
per isakson 2019-5-24
编辑：per isakson 2019-5-24
Idea :
  • Version 7.3 MAT-files are HDF5-files.
  • Matlab has functions to read HDF5-files, Importing HDF5 Files.
  • The tricky part is to find the metadata for the dataset of interest. I find HDFView useful to explore hdf-files.
  • The hdf-forum is helpful, but requires that one has done the homework before asking.
I didn't say this is simple. IMO: The format of the v7.3 MAT-file is complicated. I don't know how timeseries are stored. However, I have (years ago) made successful experiments reading datasets from v7.3 mat-files with Matlabs HDF-functions.
Walter Roberson writes:
[...] MAT 7.3 format is a complete redesign, and is not publicly documented. What is documented is that it is a variation on HDF5, and that HDF5 tools can be used to explore parts of it. MAT 7.3 is the first version that can save objects larger than 2.0 Gb.

类别

在 Help Center 和 File Exchange 中查找有关 HDF5 Files 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by