save obj creates a large mat files, disagrees with whos

Question

Julia Fischer 2019-10-30

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/488334-save-obj-creates-a-large-mat-files-disagrees-with-whos

编辑： per isakson 2019-10-31

Hello

I have a class Node with 3 properties: index , coord, and orientation; or any class.

classdef  Node
    properties 
         idx;      %index
         coord;    %coordinates
         ori;      %origin , rotation, whatever  
    end
    methods  
        function    obj=Node(id,coo,or)    %simplest construtor ever
            obj.idx=id;
            obj.coord=coo;
            obj.ori=or;
        end
    end
end

when i run this for loop:

clear nodes
for i=1:1000
    nodes(i)=Node(i,[i i i ] , [0 0 0 ]);
end

I got a vector of obj Node , and "whos nodes" gives me

Name Size Bytes Class Attributes

nodes 1x1000 56000 Node

with 7 doubles * 8 bytes/double * 1000 = 56000 Bytes

But save produces an extremendly larger file

save 'nodes.mat' nodes;

ls -la nodes.mat = 1179578 bytes ~1.2M

I would have expected the mat file to be a big larger than what is reported by whos ( size, m ,n , address, class name, etc ) but not 20 times larger

How could it be ?

Thanks

Julia

2 个评论
显示无隐藏无

per isakson 2019-10-30

编辑：per isakson 2019-10-31

在 MATLAB Online 中打开

With R2018b

save( 'nodes1.mat', 'nodes', '-v7' );
save( 'nodes2.mat', 'nodes', '-v7.3' );

The first gives a 22KB file (less than half of what whos reports) and the second 1.2MB

'How could it be?' '-v7.3' produces an involved HDF5 file. HDF5 shines when it comes to large numerical arrays and "signals".

Julia Fischer 2019-10-31

Hi Per,

thanks for you comment , but still my problem remains.

I also have other similar classes and when I have let's say 10 Mio. of objs I am over 10GB with 7.3 , instead of the 500 MB with v7. Once I have rechead the 2GB data size format v7 will no longer work. Moreover, save takes much more time using v7.3 than v 7.0

Along with my classes I have saved several large array using mat 7.3 and you are right , with HDF5 the bytes count fits the expected sizes

Since I am in a industrial enviroment we produce a lot of data very quickly .....

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

per isakson 2019-10-31

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/488334-save-obj-creates-a-large-mat-files-disagrees-with-whos#answer_399219

编辑：per isakson 2019-10-31

在 MATLAB Online 中打开

"Once I have reached the 2GB data size format v7 will no longer work." ???

Doc on save v7 says: 2^31 bytes per variable

Experiment (R2018b)

%%
a1 = rand(1e6,1e3);     % rand to avoid dramatic compression 
a01 = a1(1:1e5,:);
a02 = a1(1e5:2e5,:);
a03 = a1(2e5:3e5,:);
a04 = a1(3e5:4e5,:);
a05 = a1(4e5:5e5,:);
a06 = a1(5e5:6e5,:);
whos
  Name            Size                   Bytes  Class     Attributes
  a01        100000x1000             800000000  double              
  a02        100001x1000             800008000  double              
  a03        100001x1000             800008000  double              
  a04        100001x1000             800008000  double              
  a05        100001x1000             800008000  double              
  a06        100001x1000             800008000  double              
save( '-regexp', 'a0\d', '-v7' );

produces a 4.5GB mat-file

"industrial enviroment" You might want to discuss your problem with MathWorks. I don't know the limits of v7 and you don't want to find out when it is too late.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

save obj creates a large mat files, disagrees with whos

2 个评论
显示无隐藏无

回答（1 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

save obj creates a large mat files, disagrees with whos

2 个评论 显示 无隐藏 无

回答（1 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

2 个评论
显示无隐藏无

0 个评论
显示 -2更早的评论隐藏 -2更早的评论