Virtual Datasets Source remapping or autoupdate

1 次查看（过去 30 天）
Hoang-Le Tran
Hoang-Le Tran 2023-12-19
编辑： Hoang-Le Tran 2024-2-1
I am using Matlab's low-level hdf5 functions to map multiple chunks of an image stored in .h5 to a VDS. The problem I am having is when the vds file is moved, the mappings remain the same(expected). I was hoping see if there was a way to have the mappings be dependent on the vds file and not static to the folder from which it was mapped. Something along the line of:
Source files:/1/data_folder
VDS: /1/vds.h5
If the folder was changed to /1_new/ the mappings would be reflect based on the new location of the VDS: /1_new/vds.h5
If this isn't possible, is there a way to remap to new locations without creating a new vds or even just convert a VDS to a regular HDF5?
Edit:
Apparently, HDF5 actually has support for relative path prefix where it can be relative to the VDS itself, which might facilitate a VDS that can auto update its source locations. Documentations below. The challenge, however, is that MATLAB does not currently(as of my knowledge) have support for these low level functions, so I have not gotten a chance to try and test. HDF5 has a C library. Does anybody have an idea of how I can utilize this library in MATLAB?
H5Pset_virtual_prefix
H5Pset_efile_prefix()

请先登录，再回答此问题。

采纳的回答

Hoang-Le Tran
Hoang-Le Tran 2024-2-1
编辑：Hoang-Le Tran 2024-2-1
When mapping the VDS, it is possible to use relative pathing when using H5P.set_virtual() without using the prefix functions. As long as the provided path does not start with '/' in unix, the relative path of source files will be prepended with the current directory of the VDS file. Documentation here: H5Pset_virtual in the 'source file resolutions' section.

更多回答（1 个）

Ayush Modi
Ayush Modi 2024-1-10
Hi,
As per my understanding, you are trying to create a dynamic mapping of multiple chunks of data stored in .h5 file to VDS.
However, VDS mappings are static once created; they don't automatically update if the source files are moved to a new location. This means that if you move the source files or the VDS file to a new location, the VDS mappings will still point to the original file locations and will not reflect the new paths.
However, you can convert VDS to a regular HDF5 file. You can read the data through the VDS and write it to a new regular HDF5 file. This way, you lose the virtual mapping, but you create a standalone HDF5 file that contains all the data and does not depend on the source file locations.
Here is an example to demonstrate how you can accomplish this:
% Read the data from the VDS dataset
data = H5D.read(dset_id, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT');
% Creating a new HDF5 file and writing the data
new_file_id = H5F.create('new_regular_file.h5', 'H5F_ACC_TRUNC', 'H5P_DEFAULT', 'H5P_DEFAULT');
dataspace_id = H5S.create_simple(ndims(data), fliplr(size(data)), []);
new_dset_id = H5D.create(new_file_id, '/dataset', 'H5T_NATIVE_DOUBLE', dataspace_id, 'H5P_DEFAULT');
H5D.write(new_dset_id, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', data);
Please refer to the following MathWorks documentation for more information on the following functions:
HF5 - https://www.mathworks.com/help/matlab/ref/fileh5f.html
H5D - https://www.mathworks.com/help/matlab/ref/dataseth5d.html
I hope this resolves the issue you were facing.
  1 个评论
Hoang-Le Tran
Hoang-Le Tran 2024-1-10
Hey Ayush,
Thanks for chiming in. I just found today that HDF5 actually has support for relative path prefix where it can be relative to the VDS itself, which might facilitate a VDS that can auto update its source locations. Documentations below. The challenge, however, is that MATLAB does not currently(as of my knowledge) have support for these low level functions, so I have not gotten a chance to try and test. Any input on how I could call the whole HDF5 library within MATLAB? Maybe through a java wrapper or C Mex. I have no idea on how MATLAB supports external libraries.
H5Pset_virtual_prefix
H5Pset_efile_prefix()

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 HDF5 Files 的更多信息

标签

产品


版本

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by