HDF5 Insert an Array Field into Compound Dataset

17 次查看（过去 30 天）
Kat Lee
Kat Lee 2022-2-17
编辑： Aman 2024-1-24
Long story in short, I am trying to save a structure as compound dataset
below long pasted code is a simplified demo code, most fields in my struct are numbers, however, there is one field (see FieldArray) that is an array
function h5ex_t_cmpdWArray
fileName = 'h5ex_t_cmpd.h5';
DATASET = 'DS1';
DIM0 = 5;
ADIM0 = 10;
dims = DIM0;
wdata.FieldA =int32([1153 ; 1184 ; 1027 ; 1313]);
wdata.FieldB ={'Exterior (static)', 'Intake',...
'Intake manifold', 'Exhaust manifold'};
wdata.FieldC =[53.23; 55.12; 130.55; 1252.89];
wdata.FieldD =[24.57; 22.95; 31.23; 84.11];
% NEW: 4x10
wdata.FieldArray = [randn(1,ADIM0);randn(1,ADIM0);randn(1,ADIM0);randn(1,ADIM0)];
%
%% Create a new file using the default properties.
%
file = H5F.create (fileName, 'H5F_ACC_TRUNC',...
'H5P_DEFAULT', 'H5P_DEFAULT');
%
%Create the required data types
%
intType =H5T.copy('H5T_NATIVE_INT');
sz(1) =H5T.get_size(intType);
strType = H5T.copy ('H5T_C_S1');
H5T.set_size (strType, 'H5T_VARIABLE');
sz(2) =H5T.get_size(strType);
doubleType=H5T.copy('H5T_NATIVE_DOUBLE');
sz(3) =H5T.get_size(doubleType);
doubleType=H5T.copy('H5T_NATIVE_DOUBLE');
sz(4) =H5T.get_size(doubleType);
% array size
base_type_id = H5T.copy('H5T_NATIVE_DOUBLE');
dimsArr = size(wdata.FieldArray);
arrayType = H5T.array_create(base_type_id, fliplr(dimsArr));
type_id = H5T.copy(base_type_id);
H5T.set_size(type_id, 16);
type_size = H5T.get_size(type_id);
%foo = H5T.get_size(arrayType);
sz(5) = type_size;
%
% Computer the offsets to each field. The first offset is always zero.
%
offset(1)=0;
offset(2:5)=cumsum(sz(1:4));
%
% Create the compound datatype for memory.
%
memtype = H5T.create ('H5T_COMPOUND', sum(sz));
H5T.insert (memtype,...
'FieldA',offset(1),intType);
H5T.insert (memtype,...
'FieldB',offset(2), strType);
H5T.insert (memtype,...
'FieldC',offset(3), doubleType);
H5T.insert (memtype,...
'FieldD',offset(4), doubleType);
H5T.insert (memtype,...
'FieldArray',offset(5), type_id);
%
% Create the compound datatype for the file. Because the standard
% types we are using for the file may have different sizes than
% the corresponding native types, we must manually calculate the
% offset of each member.
%
filetype = H5T.create ('H5T_COMPOUND', sum(sz));
H5T.insert (filetype, 'FieldA', offset(1),intType);
H5T.insert (filetype, 'FieldB', offset(2), strType);
H5T.insert (filetype, 'FieldC',offset(3), doubleType);
H5T.insert (filetype, 'FieldD',offset(4), doubleType);
H5T.insert (filetype, 'FieldArray',offset(5), doubleType);
%
% Create dataspace. Setting maximum size to [] sets the maximum
% size to be the current size.
%
space = H5S.create_simple (1,fliplr( dims), []);
%
% Create the dataset and write the compound data to it.
%
dset = H5D.create (file, DATASET, filetype, space, 'H5P_DEFAULT');
H5D.write (dset, memtype, 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', wdata);
%
% Close and release resources.
%
H5D.close (dset);
H5S.close (space);
H5T.close (filetype);
H5F.close (file);
The code errored out for line:
H5D.write (dset, memtype, 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', wdata);
detail error see
Error using hdf5lib2
Attempted to transfer too many values to or from the library buffer.
Error in H5D.write (line 100)
H5ML.hdf5lib2('H5Dwrite', varargin{:});
Error in h5ex_t_cmpdWArray (line 112)
H5D.write (dset, memtype, 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', wdata);
There are few questions I am confused about
  1. how to insert array into compound datatype
  2. how to write
Appreciate a lot

请先登录，再回答此问题。

回答（1 个）

Aman
Aman 2024-1-24
编辑：Aman 2024-1-24
Hi Kate,
I see that you are trying to create a compound dataset, but while doing so, you are facing issues while writing it to an HDF5 file.
This issue arises when you allocate an inappropriate amount of dataspace. Please refer to the below example code where we have allocated two dataspaces for "sampleData" while each "sampleData" sample is of size one.
fileName = 'h5ex_t_cmpd.h5';
dataset = 'DS1';
sampleData(1).temperature = 11.32;
sampleData(2).temperature = 42.11;
file = H5F.create (fileName, 'H5F_ACC_TRUNC', 'H5P_DEFAULT', 'H5P_DEFAULT');
memtype = H5T.create ('H5T_COMPOUND', H5ML.sizeof (sampleData(1)));
H5T.insert (memtype, 'temperature',H5ML.hoffset (sampleData(1), 'temperature'),'H5T_NATIVE_DOUBLE');
filetype = H5T.create ('H5T_COMPOUND', 8 );
H5T.insert (filetype, 'temperature', 0 ,'H5T_NATIVE_DOUBLE');
space = H5S.create_simple (1,2, []);
dset = H5D.create (file, dataset, filetype, space, 'H5P_DEFAULT');
H5D.write (dset, memtype, 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', sampleData);
H5D.close (dset);
H5S.close (space);
H5T.close (filetype);
H5F.close (file);
In order to rectify this issue, you either need to make the size of each sample of "sampleData" two or allocate only one dataspace. In order to allocate only one dataspace in the below manner,.
space = H5S.create_simple (1,1, []);
I hope it helps!

类别

在 Help Center 和 File Exchange 中查找有关 HDF5 Files 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by