Performance issues with writing into matfiles

4 次查看（过去 30 天）
Peter Valent
Peter Valent 2022-2-4
评论： Srija Kethiri 2022-2-25
I am writing a simulation program that uses multiple long time series of substantial length. To avoid memory overflow I read the data from the predefined text files and store them on disk in a mat file. When writing datetime data into the matfiles I have encountered several issues with performance. The performance issues were quite significant as sometime it took less than 1s and sometimes ~300s.
Has anyone else observed similar performance issues? Could it be related to the datatime data type?
% Here is simple code snippet
dt = (datetime(1,1,1):datetime(3000,12,31))';
m = matfile('test.mat','Writable',true);
m.dt = dt;

请先登录，再回答此问题。

回答（1 个）

Srija Kethiri
Srija Kethiri 2022-2-18
Hi Peter,
You are getting this issue because of the matfile so delete the matfile before you write into it.
The attached code might be helpful:
delete('test.mat');
dt = (datetime(1,1,1):datetime(3000,12,31))';
m = matfile('test.mat','Writable',true);
m.dt = dt;
You can refer to the following link to know more about,
delete function - Delete files or objects - MATLAB delete (mathworks.com)
matfiles - Access and change variables in MAT-file without loading file into memory - MATLAB (mathworks.com)
  2 个评论
Peter Valent
Peter Valent 2022-2-18
I cannot delete the matfile before I write the datetime variable into it as the file already contains other data that will be used later on. I am using the matfile to store data that would not necessarily fit into memory.
Srija Kethiri
Srija Kethiri 2022-2-25
Hi Peter,
From my understanding you can’t delete the file as the file already contains other data. The data in matfile is large to fit in memory.
So, you can load the data present in the matfile to the variables in workspace, as the data is large, the best practice is to read and write as much data into memory as possible at a time. Otherwise, repeated file access negatively impacts the performance of your code. You can use the load function to do it. And then after deleting the file rewrite this data into the same matfile.
For more information about loading parts of variables from matfile refer to this documentation: Save and Load Parts of Variables in MAT-Files - MATLAB & Simulink (mathworks.com)
Hope this helps!

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Workspace Variables and MAT-Files 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by