How to csvwrite an array in HDFS
显示 更早的评论
Hi,
I have Matlab connected to a Hadoop cluster. I want to save an array as a csv file to HDFS. The only commend I found was:
location = 'hdfs://master/user/savecsvfile';
write(location, Array1 );
Then I check the HDFS respiratory to find a file "part-1-snapshot.seq" and the contents are different from what Array1 is.
Thanks,
采纳的回答
Kojiro Saito
2018-1-16
write command is able to write only tall array as .seq file. So, if you want to save csv files to HDFS, you should first save csv files on locla then upload to HDFS using Hadoop commands. From MATLAB,
csvwrite('hoge.csv', Array1);
!hadoop fs -copyFromLocal ./hoge.csv /user/savecsvfile
6 个评论
Amr Munshi
2018-1-16
Kojiro Saito
2018-1-16
In order to read .seq files from HDFS, we use datastore. For example,
location = 'hdfs://master/some/path/to/seqfiles';
ds = datastore(location);
Amr Munshi
2018-1-16
Kojiro Saito
2018-1-16
Do you want to create HDFS directory with timestamp? If so, try this.
formatOut = 'dd-mm-yy_HHMM';
command = ['hadoop fs -copyFromLocal ./hoge.csv /user/savecsvfile' datestr(now,formatOut)];
status = system(command);
In this case, a csv file will be stored in savecsvfile16-01-18_1410.
Amr Munshi
2018-1-17
lov kumar
2019-6-7
How to connect matlab to hadoop cluster.
How to fix this issue:
Parallel mapreduce execution on the Hadoop cluster:
********************************
* MAPREDUCE PROGRESS *
********************************
Map 0% Reduce 0%
Error using mapreduce (line 125)
The HADOOP job failed to submit. It is possible that there is some issue with the HADOOP configuration.
OS: windows
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Introduction to Installation and Licensing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
