How to best store and work with timeseries datasets in Matlab?
15 次查看(过去 30 天)
显示 更早的评论
I am currently using a trial version of Matlab and I have matlab data files that were collected for 1.5 years in separate files. Within these files the data is collected almost every second, hence there would be sometimes 700 to 900 thousand rows in each of these files. I would like to know how best I can combine these files and if I will really need to create a database to store the raw data? I would like to then visualise this data over the 1.5 year period so I do need to store them all in one file. I am not familar with setting a database so will welcome suggestions on what I can do and which tutorial I can use that can help me with setting up the db or combining the files.
Thank you.
0 个评论
回答(2 个)
Peter Perkins
2019-4-16
MSani, you probably want to take a look at datastores, and tall (or not tall) timetables.
Aylin
2019-4-17
Hi MSani, as Peter suggested you could use datastore to make it easier to work with large files and datasets that may not fit in memory.
In particular, FileDatastore can be used to load a collection of MAT files using the load function. Here's an example that loads and reads a MAT file from the MATLAB demos directory:
files = fullfile(matlabroot, "toolbox", "matlab", "demos", "trimesh" + ["2d", "3d"] + ".mat");
% A function that reads MAT file data and converts it to a table
readFcn = @(filename) struct2table(load(filename, 'x', 'y'));
% Construct a FileDatastore to read all the MAT files with the custom reading function
fds = fileDatastore(files, "ReadFcn", readFcn, "UniformRead", true);
If your data fits in memory, you could read all the MAT files at once and plot it using the stackedplot function:
data = readall(fds); % Reads all the data into memory
% Convert to the 'timetable' datatype and plot the time series data
ttData = table2timetable(data, 'SampleRate', 0.1); % Sample rate is 0.1 Hz
stackedplot(ttData)
This generates a plot that looks like this:
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/214579/image.png)
If you have data that doesn't fit into memory, you could construct a tall timetable instead. Here's an example of this workflow on the MATLAB doc.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!