HDP 3.1.1 and MATLAB R2019a Compatability
2 次查看(过去 30 天)
显示 更早的评论
I have a cluster of linux machines running HDP 3.1.1. I see in all the HDFS documentation that only Hadoop versions 1.x and 2.x are supported. Will I have compatability issues due to using a 3.x version of Hadoop? When I try to read HDFS files from the cluster from a edge Windows machine, I continually get an error when using datastore (see below).
Error using datastore (line 120)
Cannot determine the Hadoop version. Verify that HADOOP_PREFIX, HADOOP_HOME, or the MATLAB_HADOOP_INSTALL environment variable is set to the root of your Hadoop installation folder.
The Windows machine I am accessing the cluster from does not have Hadoop installed and I point the required environment variable(s) to root of the Hadoop installation on the namenode of the cluster. I have spent considerable time attempting all fixes I could find assuming that Hadoop 3.1.1 is compatable with Matlab, but am suspecting that may be the root of the problem.
-Brian
0 个评论
采纳的回答
Onomitra Ghosh
2019-10-1
Brian, can you provide some more details on the problem you are running into when running the mapreduce command. A code example of how you are setting the environment variable, creating datastore and running mapreduce will be good. Also, is the error message exactly the same as before? Is this the complete error stack?
2 个评论
Onomitra Ghosh
2019-10-2
Brian, thanks for finding this. You have uncovered an issue in the way we seek for Hadoop version. We will work on finding a better solution for this. Until then, please continue to have the HDFS installation on the worker nodes as a workaround. We will reach out to you if we need more information.
更多回答(1 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 HDF5 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!