mex error when reading lz4 compressed h5 file... sometimes

3 次查看（过去 30 天）
Thorbjørn Erik Køppen Christensen
编辑： Thorbjørn Erik Køppen Christensen 2021-3-2
I'm testing some code, and to check for bottlenecks I made three copies of my files
1) on a shared network drive
2) on a HDD next to my workstation connectet through a USB dock
3) on a ramdisk on the computer (just to see how fast the code would actually run)
I made 2) as a copy of 1) and 3) as a copy of 2) and the data is 150ish GiB
When I process the data from 1) Everything is fine.
When I process the data from 2) or 3) however, things got weird. for some files I would suddenly get workers to drop dead. It turns out this was due to an error in the MEX file for reading HDF5 files. The files that gave this error turned out not to be the same on 2) and 3), and some files affect on 3) where not affected on 2) and vice versa. By copying the files affected on 3) back to 3) from 2) I somehow succeeded in running my code from 3). But this aproach does not work for 2) for some reason.
When I replace the files matlab can read them, but when I then restart matlab, like it asks me to, it will find some other files in my dataset that it cannot read.
I haven't had this problem before?
What's even weirder is that I can read part of all the files, as I tried:
fileholder = dir('file/folder/*.h5')
for ii = numel(fileholder):-1:1
disp(ii)
filecontent = h5read(fullfile(fileholder(ii).folder,fileholder(ii).name),'/path/to/data',[1 1 1 1], [1 1 1 1])
end
I had no problems (most of the time). whereas
fileholder = dir('file/folder/*.h5')
for ii = numel(fileholder):-1:1
disp(ii)
filecontent = h5read(fullfile(fileholder(ii).folder,fileholder(ii).name),'/path/to/data')
end
does give an error
I'm getteng the error only on windows from these files (but often it will only be one file in more than 300)
The install is rather new, so I'm wondering, could it be a change in the hdf5plugin for python?
The crash dump I get looks like this:
--------------------------------------------------------------------------------
Access violation detected at 2021-03-02 17:56:03 +0100
--------------------------------------------------------------------------------
Configuration:
Crash Decoding : Disabled - No sandbox or build area path
Crash Mode : continue (default)
Default Encoding : US-ASCII
Deployed : false
Graphics Driver : Unknown hardware
Graphics card 1 : NVIDIA ( 0x10de ) NVIDIA GeForce RTX 3090 Version 27.21.14.5671 (2020-9-30)
Java Version : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
MATLAB Architecture : win64
MATLAB Entitlement ID : 1242273
MATLAB Root : C:\Program Files\Matlab R2020b
MATLAB Version : 9.9.0.1467703 (R2020b)
OpenGL : hardware
Operating System : Microsoft Windows 10 Enterprise LTSC
Process ID : 43000
Processor ID : x86 Family 143 Model 49 Stepping 0, AuthenticAMD
Session Key : 8a396e89-e08e-47dd-a8b3-685e64d67c23
Window System : Version 10.0 (Build 17763)
Fault Count: 1
Abnormal termination:
Access violation
Current Thread: 'MCR 0 interpreter thread' id 57536
Register State (from fault):
RAX = 0000019dcb3819d6 RBX = 0000019dca25cffa
RCX = 0000000000000000 RDX = 0000019dcb3819d2
RSP = 00000031a37f78c0 RBP = 0000019dc9d7d050
RSI = 0000019dcb382928 RDI = 0000000000000000
R8 = 0000000000000000 R9 = 0000000000000000
R10 = 0000019dcb3819d6 R11 = 0000000000000246
R12 = 00007ff8e8710000 R13 = 0000019dca266040
R14 = 0000019dcb38290e R15 = 0000000000000000
RIP = 00007ff8e8714249 EFL = 00010293
CS = 0033 FS = 0053 GS = 002b
Stack Trace (from fault):
[ 0] 0x00007ff8e8714249 C:\Users\Guldfaxe\AppData\Local\Programs\Python\Python39\Lib\site-packages\hdf5plugin\plugins\libh5lz4.dll+00016969 H5PLget_plugin_type+00008505
[ 1] 0x00007ff8e8711f15 C:\Users\Guldfaxe\AppData\Local\Programs\Python\Python39\Lib\site-packages\hdf5plugin\plugins\libh5lz4.dll+00007957
[ 2] 0x00007ff89dd843a8 C:\Program Files\Matlab R2020b\bin\win64\hdf5.dll+01524648 H5Z_pipeline+00000520
[ 3] 0x00007ff89dc40d7d C:\Program Files\Matlab R2020b\bin\win64\hdf5.dll+00200061 H5D__chunk_lock+00000845
[ 4] 0x00007ff89dc3c488 C:\Program Files\Matlab R2020b\bin\win64\hdf5.dll+00181384 H5D_btree_debug+00004952
[ 5] 0x00007ff89dc4cabc C:\Program Files\Matlab R2020b\bin\win64\hdf5.dll+00248508 H5D__read+00001212
[ 6] 0x00007ff89dc4d88a C:\Program Files\Matlab R2020b\bin\win64\hdf5.dll+00252042 H5Dread+00000522
[ 7] 0x00007ff8912f3236 C:\Program Files\Matlab R2020b\toolbox\matlab\imagesci\private\h5readc.mexw64+00078390 mexFunction+00001334
[ 8] 0x00007ff8b0d9a289 C:\Program Files\Matlab R2020b\bin\win64\libmex.dll+00565897 mexfile::Inspector::needs_upgrade+00015977
[ 9] 0x00007ff8b0d9b67b C:\Program Files\Matlab R2020b\bin\win64\libmex.dll+00571003 MexRetrieveVersion+00003579
[ 10] 0x00007ff8b0d9b9b4 C:\Program Files\Matlab R2020b\bin\win64\libmex.dll+00571828 MexRetrieveVersion+00004404
[ 11] 0x00007ff8b0d7c209 C:\Program Files\Matlab R2020b\bin\win64\libmex.dll+00442889 mexUnlock_800+00023993
[ 12] 0x00007ff8b0c726c1 bin\win64\pgo\m_dispatcher.dll+00665281 Mfh_file::dispatch_file_common+00000165
[ 13] 0x00007ff8b0c71d62 bin\win64\pgo\m_dispatcher.dll+00662882 Mfh_error::lookup+00006498
[ 14] 0x00007ff8b0c72615 bin\win64\pgo\m_dispatcher.dll+00665109 Mfh_file::dispatch+00000045
[ 15] 0x00007ff8ae318208 bin\win64\pgo\m_lxe.dll+00557576 CGGBuiltinImpl+00181784
[ 16] 0x00007ff8ae2fb484 bin\win64\pgo\m_lxe.dll+00439428 CGGBuiltinImpl+00063636
[ 17] 0x00007ff8ae417861 bin\win64\pgo\m_lxe.dll+01603681 mwboost::serialization::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >+00649937
[ 18] 0x00007ff8ae2ff20d bin\win64\pgo\m_lxe.dll+00455181 CGGBuiltinImpl+00079389
[ 19] 0x00007ff8ae2fe4dd bin\win64\pgo\m_lxe.dll+00451805 CGGBuiltinImpl+00076013
[ 20] 0x00007ff8ae2ed187 bin\win64\pgo\m_lxe.dll+00381319 CGGBuiltinImpl+00005527
[ 21] 0x00007ff8ae2ecd8f bin\win64\pgo\m_lxe.dll+00380303 CGGBuiltinImpl+00004511
[ 22] 0x00007ff8ae2f8b2c bin\win64\pgo\m_lxe.dll+00428844 CGGBuiltinImpl+00053052
[ 23] 0x00007ff8ae2fd185 bin\win64\pgo\m_lxe.dll+00446853 CGGBuiltinImpl+00071061
[ 24] 0x00007ff8ae2fc19c bin\win64\pgo\m_lxe.dll+00442780 CGGBuiltinImpl+00066988
[ 25] 0x00007ff8ae412c1f bin\win64\pgo\m_lxe.dll+01584159 mwboost::serialization::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >+00630415
[ 26] 0x00007ff8ae412b07 bin\win64\pgo\m_lxe.dll+01583879 mwboost::serialization::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >+00630135
[ 27] 0x00007ff8ae412a16 bin\win64\pgo\m_lxe.dll+01583638 mwboost::serialization::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >::singleton<mwboost::serialization::extended_type_info_typeid<MathWorks::lxe::PreLineExecutionEvent> >+00629894
[ 28] 0x00007ff8ae452d7c bin\win64\pgo\m_lxe.dll+01846652 mwboost::archive::detail::oserializer<mwboost::archive::binaryTerm_oarchive,MathWorks::lxe::ClosedAbstractionFunctionDescriptorOwningIrTree>::oserializer<mwboost::archive::binaryTerm_oarchive,MathWorks::lxe::ClosedAbstractionFunctionDescriptorOwningIrTree>+00024116
[ 29] 0x00007ff8ae452d2a bin\win64\pgo\m_lxe.dll+01846570 mwboost::archive::detail::oserializer<mwboost::archive::binaryTerm_oarchive,MathWorks::lxe::ClosedAbstractionFunctionDescriptorOwningIrTree>::oserializer<mwboost::archive::binaryTerm_oarchive,MathWorks::lxe::ClosedAbstractionFunctionDescriptorOwningIrTree>+00024034
[ 30] 0x00007ff8afde69ba bin\win64\pgo\m_lxe.dll+28666298 MathWorks::lxe::profiler::IMexProfilingSupport::~IMexProfilingSupport+00058926
[ 31] 0x00007ff8cbcd6549 C:\Program Files\Matlab R2020b\bin\win64\libmwbridge.dll+00157001 mnGetPrompt+00016921
[ 32] 0x00007ff8b0e911ff C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00463359 iqm::Iqm::instance+00001023
[ 33] 0x00007ff8b0f038b7 C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00932023 iqm::UserEvalPlugin::execute+00001351
[ 34] 0x00007ff8a4fe80be C:\Program Files\Matlab R2020b\bin\win64\nativejmi.dll+00360638 nativejmi::JmiIIP::serializeExplicit+00013438
[ 35] 0x00007ff8b0eb7a3f C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00621119 iqm::Iqm::setupIqmFcnPtrs+00113615
[ 36] 0x00007ff8b0ec31b0 C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00668080 iqm::Iqm::setupIqmFcnPtrs+00160576
[ 37] 0x00007ff8b0e86fc3 C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00421827 iqm::Iqm::create+00007235
[ 38] 0x00007ff8cbcc7a3e C:\Program Files\Matlab R2020b\bin\win64\libmwbridge.dll+00096830 ioReadLine+00000478
[ 39] 0x00007ff8cbcc781c C:\Program Files\Matlab R2020b\bin\win64\libmwbridge.dll+00096284 ioReadLine+00000188
[ 40] 0x00007ff8cbcd6930 C:\Program Files\Matlab R2020b\bin\win64\libmwbridge.dll+00158000 mnGetCommandLineBuffer+00000304
[ 41] 0x00007ff8cbcd6db4 C:\Program Files\Matlab R2020b\bin\win64\libmwbridge.dll+00159156 mnParser+00000500
[ 42] 0x00007ff8b1056302 C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00549634 mcr_set_enableReadingFromStdin+00016098
[ 43] 0x00007ff8b0fee7ad C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00124845 mcrFunctionSignature::set_signature+00103437
[ 44] 0x00007ff8b100baea C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00244458 mcrFunctionSignature::set_signature+00223050
[ 45] 0x00007ff8b0ee4504 C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00804100 iqm::PackagedTaskPlugin::execute+00000084
[ 46] 0x00007ff8b10503ce C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00525262 services::lmgr::exception::LicensingStartupException::~LicensingStartupException+00009822
[ 47] 0x00007ff8b0eb7a3f C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00621119 iqm::Iqm::setupIqmFcnPtrs+00113615
[ 48] 0x00007ff8b0e87c42 C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00425026 iqm::Iqm::create+00010434
[ 49] 0x00007ff8b0e87421 C:\Program Files\Matlab R2020b\bin\win64\iqm.dll+00422945 iqm::Iqm::create+00008353
[ 50] 0x00007ff8b10260dc C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00352476 mcrInstantiationError::operator=+00011756
[ 51] 0x00007ff8b1026a3c C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00354876 mcrInstantiationError::operator=+00014156
[ 52] 0x00007ff8b10243cc C:\Program Files\Matlab R2020b\bin\win64\mcr.dll+00345036 mcrInstantiationError::operator=+00004316
[ 53] 0x00007ff8e959a2a3 C:\Program Files\Matlab R2020b\bin\win64\mwboost_thread-vc141-mt-x64-1_70.dll+00041635 mwboost::thread::swap+00000083
[ 54] 0x00007ff8f16e268a C:\Windows\System32\ucrtbase.dll+00140938 o_exp+00000090
[ 55] 0x00007ff8f4317974 C:\Windows\System32\KERNEL32.DLL+00096628 BaseThreadInitThunk+00000020
[ 56] 0x00007ff8f471a2d1 C:\Windows\SYSTEM32\ntdll.dll+00369361 RtlUserThreadStart+00000033
This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 External Language Interfaces 的更多信息

标签

产品


版本

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by