Reading in csv files

Question

Seth 2013-7-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/82267-reading-in-csv-files

编辑： Walter Roberson 2018-10-13

I have a bit of program to read in csv files but it takes a really long time, most likely because of the "for" loops. Can anyone inform me on ways I can make this program more efficient?

%
%  Load and convert a csv file
%
%clear all contents
clear
clc
tic;
%read in file
fileName = 'SmartBowAccelTrunk.csv';
%fileName = 'testDoc.txt';
%the order of the file is as follows
%timestamp;SBname;farmid;tagmac;xval;yval;zval;modul;soft;accX;accY;accZ;absAcc
%20130625182723252;xxxx;182;000008B1;11.01;4.91;;h;5;-71;-583;831;1017.591
%d  %s %d %s %f %f %f %s %d %d %d %d %d %f
%read in csv file as txt
[num, txt, raw] = xlsread(fileName);  %returns a cell array of strings
%first row is header information read in as string %s
header = textscan(txt{1}, '%s%s%s%s%s%s%s%s%s%s%s%s%s', 'Delimiter', ';');  %13 columns of data... read each of the header titles
header = [header 'time' 'hour' 'minute' 'second' 'ms'];
% loop through remaining rows and extract data (first as text, then as relevant data format)
for iRow=2:length(txt),
    %rowData = textscan(txt{iRow}, '%f%s%f%s%f%f%f%s%d%d%d%d%f', 'Delimiter', ';');
    rowData = textscan(txt{iRow},'%s%s%s%s%s%s%s%s%s%s%s%s%s','Delimiter', ';');  %just grab as strings, convert later
    for iCol=1:length(rowData)
        allData(iRow-1,iCol) = rowData{iCol};
    end
    %change time string to number
     allTimes(iRow-1) = allData(iRow-1,1);
   %the number to be parsed
a=allTimes';
mstimes=rem(a, 1000000000);
tn=datenum(a,'yyyymmddHHMMSSFFF');
time_hh=str2num(datestr(tn,'HH'));
time_mm=str2num(datestr(tn,'MM'));
time_ss=str2num(datestr(tn,'SS'));
time_ms=str2num(datestr(tn,'FFF'));
end
%add row with time
   %allData = [allData times];
cellTime=num2cell(mstimes);
cellHour=num2cell(time_hh);
cellMin=num2cell(time_mm);
cellSec=num2cell(time_ss);
cellMs=num2cell(time_ms);
allData=[allData cellTime cellHour cellMin cellSec cellMs];
allData=sortrows(allData,[3 14]);
output={header; allData};

toc

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Image Analyst 2013-7-16

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/82267-reading-in-csv-files#answer_91950

And the reason you're not using csvread() to read in your csv file is what?????

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

Seth 2013-7-17

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/82267-reading-in-csv-files#answer_92010

I'm very bad at Matlab. I just started using it and I'm kind of piecing together what I can from one introductory book. Does csvread() work a lot better?

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 3

Don 2018-10-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/82267-reading-in-csv-files#answer_341139

编辑：Walter Roberson 2018-10-13

在 MATLAB Online 中打开

[file, path] = uigetfile('*.csv');
file_name = strcat(path, file);
csv_file = fopen(file_name);
tic;
line = fgetl(csv_file);
% Get the names form the first line
names = textscan(line, '%s');
% this reads 90,002 lines  with 20 fields in 4.228 seconds
cntr = 1;
% make a cell to get started
data = cell(10,1);
while ~feof(csv_file)
  line2 = fgetl(csv_file);
  % Skip the first 2 columns, I don't wnat tme, take this out if you wnat
  % them
  [DateTime, pos] = textscan(line2, '%s', 2, 'Delimiter', ',');
  len = size(line2);
  data{cntr} = textscan(line2(pos+1:len(2)), '%f', 'Delimiter', ',');
  cntr = cntr+1;
end
fprintf('Read %d lines\n', cntr);
fclose(csv_file);
toc