MatFile grows exponentially larger in an empty table replacement loop?

Question

埃博拉酱 2020-5-18

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/526294-matfile-grows-exponentially-larger-in-an-empty-table-replacement-loop

回答： Asvin Kumar 2020-5-20

在 MATLAB Online 中打开

First create a MatFile:

a=matfile("Test.mat");

Then create an empty table:

a.Data=table('Size',[0 1],'VariableTypes',"uint8");

The Test.mat is 23.3 ㎅. Now we run the code again to replace the table with the same new one:

a.Data=table('Size',[0 1],'VariableTypes',"uint8");

Note the file becomes 51.6 ㎅, about 2.2 times larger. Again, 91.6 ㎅, 1.8 times larger; 150 ㎅, 1.6 times larger; 223㎅, 1.5 times larger; 308㎅, 1.4 times larger; 407 ㎅, 1.3 times larger … What a horrible increase!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Asvin Kumar 2020-5-20

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/526294-matfile-grows-exponentially-larger-in-an-empty-table-replacement-loop#answer_434788

This is a known issue in MAT files saved with version 7.3 based on HDF5 which does not manage freespace as effectively as it should. Frequently rewriting or deleting a dataset can result in freespace and cause the file to grow unnecessarily large.

One workaround is to avoid changing a single variable in the matfile handle. Instead, load the variables and save it into a new MAT file. The other workaround is to choose the format 7.0 by using '-v7' argument of save. Please refer to the following section:

https://www.mathworks.com/help/matlab/ref/save.html#inputarg_version

Some useful links containing the detailed explanation:

https://stackoverflow.com/questions/41526334/matlab-matfile-increases-in-size-when-overwriting-cell-data

https://support.hdfgroup.org/HDF5/doc/H5.user/Performance.html