Cannot import/load/read complete .dat file

5 次查看(过去 30 天)
Hi
I am trying to open my .dat file that has 785663 rows x 29 columns; each column separated by a comma (therefore, csv). I tried using textscan, importdata, csvread, load, etc in order to access my files and hence do some data analysis. The first 2 lines of my .dat file (excluding the headerline) look like this:
"2013-01-02 21:10:00",64359,18.33,20.41257,19.95976,11.60755,4.172891,56.92463,35.29119,10.93861,0,0,0,0,2.630428,0.4060569,0.7230025,0.3542804,-0.0027175,0.2566106,2.75231,2.727983,114.6311,15.36885,7.615118,99.7831,0,23.01129,12.45
"2013-01-02 21:15:00",64360,18.27,20.36502,19.92328,11.72347,4.227652,57.53248,35.50731,10.93866,0,0,0,0,2.427403,0.3657977,0.7309517,0.3855992,0.03224583,0.2124791,2.561898,2.535069,113.2416,16.75837,8.289121,99.78176,0,22.88334,12.47
The first column is date and the rest is all numeric.
I tried using the following code:
fid = fopen(filename,'r');
formatspec = ['%s' repmat('%f',1,29)];
A = textscan(fid, formatspec, 'HeaderLines',4,'Delimiter',',' ,inf, 'CollectOutput', true);
But my A cell array returns me only 11184 rows in some columns, and 11183 rows in other columns.
I also tried importdata, and again, it gave me only 11184 rows for all the columns which are numeric except the date column (11188 rows: because include the headerlines which are text).
In anyway, I couldn't get all my 785663 rows loaded. How can I get the remaining part of my dataset?
Thank you.
  1 个评论
Stephen23
Stephen23 2014-12-1
编辑:Stephen23 2014-12-1
Did you look at rows 11184/11185? Is it possible that the data on these rows might be incomplete or not able to be parsed with either of the format specifiers that you used?
When I run the code I get an error "??? Error using ==> textscan Param/value pairs must come in pairs." This seems to come from the inf value which is entered as one of the textscan options. Removing this inf gives me a 1x2 cell... but with empty cells, due to not having header lines :)
I made some small changes to the code you supplied (particularly the string format), and this imports the data correctly:
filename = 'temp.txt';
fid = fopen(filename,'r');
formatspec = ['%q',repmat('%f',1,28)];
A = textscan(fid, formatspec, 'Delimiter',',', 'CollectOutput', true);
fclose(fid);
It is likely to be something specific to the rows where these formats do not parse the data...
Note that using importdata also works for me:
B = importdata(filename, ',');

请先登录,再进行评论。

回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Large Files and Big Data 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by