Performance problems: using Msgpack library with MATLAB
7 次查看(过去 30 天)
显示 更早的评论
msgpack is a high-performance serialisation library used in many 3rd-party products. There are native interfaces available for many languages, except for MATLAB. For MATLAB there is a interpreted solution: https://github.com/bastibe/matlab-msgpack — but for my use case (deserialising messages for an eyetyracker used for neuroscience experiments), it takes ~4ms which is not viable when our experiment display loop is 120Hz (8ms). In Python the same deserialisation takes around 0.1ms. There is a C library: https://github.com/msgpack/msgpack-c — but when I try to load it in MATLAB I get the following errors:
>> [n,w]=loadlibrary('/Users/ian/Downloads/msgpack-x86/lib/libmsgpack-c.dylib','/Users/ian/Downloads/msgpack-x86/include/msgpack.h','alias','msgpack')
Warning: No functions found in library.
> In loadlibrary
n =
1×0 empty cell array
w =
'
Failed to parse type 'union { char *__mbstate8; long long _mbstateL ; } __mbstate_t' original input 'union { char __mbstate8 [ 128 ]; long long _mbstateL ; } __mbstate_t'
Found on line 67 of input from line 81 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/i386/_types.h
Type '__mbstate_t' was not found. Defaulting to type error.
Found on line 69 of input from line 83 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/i386/_types.h
Type 'longdouble' was not found. Defaulting to type error.
Found on line 1870 of input from line 16 of file /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/14.0.3/include/__stddef_max_align_t.h
Type 'sockaddrPtr' was not found. Defaulting to type voidPtr.
Found on line 2209 of input from line 317 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/sys/socket.h
Type 'sockaddrPtr' was not found. Defaulting to type voidPtr.
Found on line 2209 of input from line 317 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/sys/socket.h
No match found for enum value expression sizeof in (( sizeof ( __int64_t )) - sizeof ( __uint8_t ) - sizeof ( sa_family_t )) expression ignored.
Found on line 2257 of input from line 470 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/sys/socket.h
No match found for enum value expression sizeof in ( 128 - sizeof ( __uint8_t ) - sizeof ( sa_family_t ) - (( sizeof ( __int64_t )) - sizeof ( __uint8_t ) - sizeof ( sa_family_t )) - ( sizeof ( __int64_t ))) expression ignored.
Found on line 2257 of input from line 470 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/sys/socket.h
Type 'in6_addr' was not found. Defaulting to type error.
Found on line 2438 of input from line 177 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/netinet6/in6.h
Type 'in6_addr' was not found. Defaulting to type error.
Found on line 2451 of input from line 540 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/netinet6/in6.h
Type 'in6_addr' was not found. Defaulting to type error.
Found on line 2459 of input from line 548 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/netinet6/in6.h
Function pointer types are unsupported in structures fpos_t (* _Nullable _seek ) ( void *, .
Found on line 2730 of input from line 157 of file /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk/usr/include/_stdio.h
Failed to parse type 'union { _Bool boolean ; uint64_t u64 ; int64_t i64 ; double f64 ; msgpack_object_array array ; msgpack_object_map map ; msgpack_object_str str ; msgpack_object_bin bin ; msgpack_object_ext ext ; } msgpack_object_union' original input 'union { _Bool boolean ; uint64_t u64 ; int64_t i64 ; double f64 ; msgpack_object_array array ; msgpack_object_map map ; msgpack_object_str str ; msgpack_object_bin bin ; msgpack_object_ext ext ; } msgpack_object_union'
Found on line 2990 of input from line 88 of file /Users/ian/Downloads/msgpack-x86/include/msgpack/object.h
Type 'msgpack_object_union' was not found. Defaulting to type error.
Found on line 2995 of input from line 93 of file /Users/ian/Downloads/msgpack-x86/include/msgpack/object.h
'
Does anyone with C experience know if this is easy to workaround or not?
1 个评论
Christopher Nadler
2024-6-4
I have the same performance problems. Also tried to convert the deserialized data from python to matlab over Core_py2matlab, saving directly from python with scipy.io.savemat and loading th file in matlab, trying to use msgpack-python directly in matlab without succesfully deserializing my data, but every aproach takes to long or is not possible to realize without a substancial effort.
I also tried to compile https://github.com/bastibe/matlab-msgpack into a mex-function, but the code is not ready for this (needs to be fully readapted).
So, If someone has a performat msgparser for matlab, I would be more than happy (and Ian maybe also) =)
回答(1 个)
Christopher Nadler
2024-6-17
编辑:Christopher Nadler
2024-6-17
I found the bottleneck in https://github.com/bastibe/matlab-msgpack. I'm attemping to push a pull-request in the repo.
In my special case I could reduce the mean parse time from ~40ms to ~6ms by making additional adaptations to the code I posted. I am receiving data every ~4ms, so I am still not in real-time, but at least near enought.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Call Python from MATLAB 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!