Run Matlab on Cloudera CDH spark hangs

Question

wei li 2019-12-19

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/497350-run-matlab-on-cloudera-cdh-spark-hangs

在 MATLAB Online 中打开

I try run matlab spark example on cdh, but it hangs forever.

version:

matlab 2019b

Matlab runtime 9.7

Cloudera cdh 5.13.3

hadoop 2.6.0-cdh5.13.3

spark 1.6.0

I find it hangs on /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/ctfserver command, this is this command gdb info:

(gdb) thread apply all bt
Thread 3 (Thread 0x7f373dc40700 (LWP 6841)):
#0  0x00007f374e28870d in read () from /lib64/libpthread.so.0
#1  0x00007f3749c005cf in mwboost::iostreams::detail::file_descriptor_impl::read(char*, long) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwboost_iostreams.so.1.65.1
#2  0x00007f374d57d247 in mwboost::iostreams::detail::indirect_streambuf<mwboost::iostreams::file_descriptor_source, std::char_traits<char>, std::allocator<char>, mwboost::iostreams::input_seekable>::underflow() ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmst.so
#3  0x00007f374dfee359 in std::istream::sentry::sentry(std::istream&, bool) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6
#4  0x00007f374dfefe2f in std::istream& std::istream::_M_extract<long>(long&) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6
#5  0x00007f374d589831 in foundation::msg_svc::transport::sync::IOStreamTransport::readImpl(mwboost::shared_ptr<std::istream> const&, bool)
    () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmst.so
#6  0x00007f374d58b5c9 in foundation::msg_svc::transport::sync::IOStreamTransport::read() ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmst.so
#7  0x00007f374d58d1ee in foundation::msg_svc::transport::sync::SynchronousPostalCarrier::Runnable::run() ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmst.so
#8  0x00007f374e4a9bdd in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwboost_thread.so.1.65.1
#9  0x00007f374e281e25 in start_thread () from /lib64/libpthread.so.0
#10 0x00007f374da1734d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f372f8a6700 (LWP 6847)):
#0  0x00007f374da117f9 in syscall () from /lib64/libc.so.6
#1  0x00007f374b851744 in futex_wait (futex=<optimized out>, comparand=<optimized out>) at ../../include/tbb/machine/linux_common.h:60
#2  P (this=<optimized out>) at ../../src/tbb/semaphore.h:206
#3  commit_wait (this=<optimized out>, thr=...) at ../../src/tbb/concurrent_monitor.h:144
#4  tbb::internal::concurrent_queue_base_v3::internal_pop (this=0x7f372f8a5b58, dst=0x0) at ../../src/tbb/concurrent_queue.cpp:457
#5  0x00007f374d8769e6 in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwms.so
#6  0x00007f373c9dd022 in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwMVMLocal.so
#7  0x00007f373c9cd607 in mwboost::detail::thread_data<mwboost::function0<void> >::run() ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwMVMLocal.so
#8  0x00007f374e4a9bdd in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwboost_thread.so.1.65.1
#9  0x00007f374e281e25 in start_thread () from /lib64/libpthread.so.0
#10 0x00007f374da1734d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f374432b900 (LWP 6811)):
#0  0x00007f374e285945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f374f18117d in mwboost::condition_variable::wait(mwboost::unique_lock<mwboost::mutex>&) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmvm.so
#2  0x00007f374e4aaa54 in mwboost::thread::join_noexcept() ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwboost_thread.so.1.65.1
#3  0x00007f374d2e4428 in mwboost::thread::join() () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwfl.so
#4  0x00007f373c9c1b8d in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwMVMLocal.so
#5  0x00007f374f1e99ec in mvm::detail::SessionImpl::mvmServiceLoop(bool) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmvm.so
#6  0x000000000040b42e in ?? ()
#7  0x000000000040dfc3 in ?? ()
#8  0x00007f374f1e9acc in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmvm.so
#9  0x00007f373c9cd587 in mwboost::detail::function::function_obj_invoker0<mwboost::_bi::bind_t<mwboost::_bi::unspecified, mwboost::function1<int, std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&>, mwboost::_bi::list1<mwboost::_bi::value<std::vec---Type <return> to continue, or q <return> to quit---
tor<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > > > >, int>::invoke(mwboost::detail::function::function_buffer&) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwMVMLocal.so
#10 0x00007f373c75f487 in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmcr.so
#11 0x00007f373c75f9c8 in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmcr.so
#12 0x00007f373c75fc74 in mcr_run_main(mwboost::function0<int> const&, bool, bool) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmcr.so
#13 0x00007f373c9c22e4 in ?? () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwMVMLocal.so
#14 0x00007f374f1ea1d7 in mvm::detail::SessionImpl::runMain(mwboost::function1<int, std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&> const&, std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&) ()
   from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmvm.so
#15 0x00007f374f1e9749 in mvm::detail::SessionImpl::runSessionMain(mwboost::function2<int, mwboost::shared_ptr<mvm::Session>, std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&> const&, std::vector<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >, std::allocator<std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > > > const&) () from /usr/local/MATLAB/MATLAB_Runtime/v97/bin/glnxa64/libmwmvm.so
#16 0x000000000040affe in ?? ()
#17 0x00007f374d940c05 in __libc_start_main () from /lib64/libc.so.6
#18 0x000000000040b23a in ?? ()