Find last row of csv contents using fopen, fgetl?? Or, another way?

17 次查看（过去 30 天）
Louise Wilson
Louise Wilson 2020-8-26
评论： Louise Wilson 2020-8-26
Hello, I have a large csv ~350,000 rows, which contains a lot of information I don't need. The purpose of my script is to extract the columns I need, and to tidy up or convert it to a suitable format. Since the file is large, I read it in row by row. I run this script on a loop on multiple .csv files.
The problem I am having is how to terminate each loop when it reaches the end of the .csv row, when the last row of information is read. At the moment I am using two if/break/end commands to check if the subsequent row contains char or isempty. Using isempty() by itself was not enough as one time I ran the code, I got a non-useful char value (-1) in the last row (tline) for some reason and so the while loop kept going, even though there was no data to be read.
At the moment this code below is working but I get warnings every 1-2 .csv files that
Warning: 'Inputs must be character vectors, cell arrays of character vectors, or string arrays'
Is it possible to improve this code and make it more robust? Perhaps there is a clearer way to identify the last row of data? Each .csv is a different size and I have hundreds.
Thanks!
%% AIS_conv %%
% Code to coerce AIS to numeric array %
% MATLAB datenum | lat (dec deg) | lon (dec deg) | MMSI%
dd = 'input_data';
nowd = cd; %current folder
cd(dd); %go to input folder
d = dir('*.csv');
cd(nowd) %GO BACK TO current folder
for j = 1:length(d) %for every .csv file in folder
tic %(start timer)
filename = d(j).name; %get filename
disp(filename); %display filename
try
fid = fopen(fullfile(dd,filename)); %open file (saves time to read file
%line by line, as opposed to csvread)
tline = fgetl(fid); %read first line (column headers)
i=1; %relevant column of new vectors we are adding data to
%% for each csv file convert mmsi,lat,lon and datenum
while(1) %while in particular .csv file
try
tline = fgetl(fid); %read line
tline = strrep(tline, '"', ' '); %remove "
if(ischar(tline) == false) %terminate loop once last row of .csv is reached
break; %(two checks for this as one wasn't enough!)
end
c = strsplit(tline,','); %separate columns by ,
if( isempty(c{1}) == true ) %terminate loop once last row of .csv is reached
break;
end
mmsi(i) = str2double(c{1}); %get mmsi
lon = str2double(c{3}); %convert lon to decimal degrees
londeg = floor(lon./100);
lonmin = (100*(lon./100 - floor(lon./100)));
londecdeg(i) = londeg + lonmin/60;
lat = str2double(c{2}); %convert lat to decimal degrees
lat = -lat;
latdeg = floor(lat./100);
latmin = 100*(lat./100 - floor(lat./100));
latdecdegnonvec = latdeg + latmin/60;
latdecdeg(i) = -latdecdegnonvec;
try
T(i) = datenum(c{11},'yyyy-mm-dd HH:MM:SS'); %get datetime
catch
T(i) = NaN;
end
i=i+1; %move on to next row of output
catch
disp('while loop error');
end
end
%% Create Output
out = [T' latdecdeg' londecdeg' mmsi']; %concatenate output
%Write output to file
month=strsplit(nowd,'\'); %get month to name specific file
month=month(3);
outFileName = char(strcat('DATENUM_LAT_LON_MMSI_',month,'_',num2str(j),'.csv'));
dlmwrite(fullfile(nowd,'output',outFileName),out,'precision',15);
catch
disp(['ERROR!!!!!! ' filename])
end
toc %end timer
end

请先登录，再回答此问题。

采纳的回答

Stephen23
Stephen23 2020-8-26
Use feof like this:
while ~feof(fid);
tline = fgetl(fid);
...
end

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Data Type Conversion 的更多信息

标签

产品


版本

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by