清空筛选条件
清空筛选条件

Saving many variables as NetCDF makes file too large. How to reduce?

6 次查看（过去 30 天）
Macarena
Macarena 2018-3-28
编辑： Macarena 2018-4-2
Hello all,
I would REALLY appreciate any help on this, as I looked online but didn't find straight answers. I am trying to save many variables of different sizes as NetCDF format in this way:
numrow = 576;
numcol = 48;
numlayers = 125;
numtime_save = numtime + 1; %This is usually a value of '8'
%%Define the dimensions
netcdf.setDefaultFormat('NC_FORMAT_CLASSIC') ;
ncid = netcdf.create(files_name,'NC_WRITE');
dimidrow = netcdf.defDim(ncid,'rows',numrow);
dimidcol = netcdf.defDim(ncid,'length',numcol);
dimidlay = netcdf.defDim(ncid,'layers',numlayers);
dimidtime = netcdf.defDim(ncid,'time',numtime_save);
%%Define the name of variables, with dimensions
%%Question: do these all really have to be double precision???
varmask = netcdf.defVar(ncid,'cloudmask','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_sigma = netcdf.defVar(ncid,'cloudmask_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_clear = netcdf.defVar(ncid,'cloudmask_clear','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_clear_sigma = netcdf.defVar(ncid,'cloudmask_clear_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_radar = netcdf.defVar(ncid,'cloudmask_radar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_radar_sigma = netcdf.defVar(ncid,'cloudmask_radar_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_both = netcdf.defVar(ncid,'cloudmask_both','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_both_sigma = netcdf.defVar(ncid,'cloudmask_both_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_lidar = netcdf.defVar(ncid,'cloudmask_lidar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_lidar_sigma = netcdf.defVar(ncid,'cloudmask_lidar_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varheight = netcdf.defVar(ncid,'height','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varheight_sigma = netcdf.defVar(ncid,'height_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varN = netcdf.defVar(ncid,'N','NC_BYTE',[dimidrow dimidcol dimidtime]);
%%end the variable naming and dimensions, then actually input
%%variables into netcdf file, then close the netcdf file
netcdf.endDef(ncid);
netcdf.putVar(ncid,varmask,cloudmask);
netcdf.putVar(ncid,varmask_sigma,cloudmask_sigma);
netcdf.putVar(ncid,varmask_clear,cloudmask_clear);
netcdf.putVar(ncid,varmask_clear_sigma,cloudmask_clear_sigma);
netcdf.putVar(ncid,varmask_radar,cloudmask_radar);
netcdf.putVar(ncid,varmask_radar_sigma,cloudmask_radar_sigma);
netcdf.putVar(ncid,varmask_both,cloudmask_both);
netcdf.putVar(ncid,varmask_both_sigma,cloudmask_both_sigma);
netcdf.putVar(ncid,varmask_lidar,cloudmask_lidar);
netcdf.putVar(ncid,varmask_lidar_sigma,cloudmask_lidar_sigma);
netcdf.putVar(ncid,varheight,height);
netcdf.putVar(ncid,varheight_sigma,height_sigma);
netcdf.putVar(ncid,varN,N);
netcdf.close(ncid);
clear mex
However, when I save it this way, the file size is 1.33 GB! That is huge...If I save the files as:
save(filename, 'cloudmask','cloudmask_sigma','cloudmask_clear','cloudmask_clear_sigma',...
'cloudmask_radar','cloudmask_radar_sigma','cloudmask_both','cloudmask_both_sigma',...
'cloudmask_lidar','cloudmask_lidar_sigma','height','height_sigma','N');
this file has a size of 16 MB. I would REALLY like to save in NetCDF format, because my boss needs it this way. I have many files to save, this is just an example but it is inside of a loop. PLEASE HELP!
  5 个评论
显示 3更早的评论
Macarena
Macarena 2018-3-28
编辑：Walter Roberson 2018-3-28
This worked! Thank you! I did this:
example:
comp = 5; %compression size
varmask_radar = netcdf.defVar(ncid,'cloudmask_radar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
netcdf.defVarDeflate(ncid,varmask_radar,true,true,comp);
on all the variables, and now my file has been reduced to 6.3MB!!
Walter Roberson
Walter Roberson 2018-3-28
You should post that as an Answer so we can vote for it. (You did the work of putting together the answer that would be of use to other people.)

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Macarena
Macarena 2018-4-2
编辑：Macarena 2018-4-2
%Example for one variable. Have to do on all
numrow = 576;
numcol = 48;
numlayers = 125;
numtime_save = numtime + 1; %This is usually a value of '8'
comp = 5; %compression size, for example
%%Define the dimensions
netcdf.setDefaultFormat('NC_FORMAT_CLASSIC') ;
ncid = netcdf.create(files_name,'NC_WRITE');
dimidrow = netcdf.defDim(ncid,'rows',numrow);
dimidcol = netcdf.defDim(ncid,'length',numcol);
dimidlay = netcdf.defDim(ncid,'layers',numlayers);
dimidtime = netcdf.defDim(ncid,'time',numtime_save);
varmask_radar = netcdf.defVar(ncid,'cloudmask_radar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
netcdf.defVarDeflate(ncid,varmask_radar,true,true,comp);
netcdf.endDef(ncid);
netcdf.putVar(ncid,varmask_radar_sigma,cloudmask_radar_sigma);
netcdf.close(ncid);
clear mex

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 NetCDF 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by