readData() function error message when using mapreduce

Question

Daniel Pinto 2019-8-5

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/474931-readdata-function-error-message-when-using-mapreduce

评论： Daniel Pinto 2019-8-6

I have a very large file (100's of millions of data points) and I am using mapreduce and datastore to get access to specifc chunks of data.

I get the following error (below) when I run it and I do not know how to go around it. lAny suggestions on how to deal with this?

Error using matlab.io.datastore.TabularTextDatastore/readData (line 81)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1136, field number 15) ==>
AAC,A,0,116,COM,7.63,25,24597\n
Error in matlab.io.datastore.TabularDatastore/read (line 120)
            [t, info] = readData(ds);
Error in matlab.io.datastore.internal.shim.read (line 7)
    [varargout{1 : nargout}] = read(ds);
Error in mapreduce (line 116)
    outds = execMapReduce(mrcer, ds, mapfun, reducefun, parsedStruct);
Caused by:
    Reading the variable name 'ticker' using format '%f' from file

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jeremy Hughes 2019-8-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/474931-readdata-function-error-message-when-using-mapreduce#answer_386285

ds.TextscanFormats{15} = '%q'

That will at least get you past the one line that's causing the problem here. For a more robust answer, I'd have to look at the file. It could be that there's an error in a line somewhere that's throwing off the parsing.