Reading in csv files

2 次查看（过去 30 天）
Seth
Seth 2013-7-16
编辑： Walter Roberson 2018-10-13
I have a bit of program to read in csv files but it takes a really long time, most likely because of the "for" loops. Can anyone inform me on ways I can make this program more efficient?
%
% Load and convert a csv file
%
%clear all contents
clear
clc
tic;
%read in file
fileName = 'SmartBowAccelTrunk.csv';
%fileName = 'testDoc.txt';
%the order of the file is as follows
%timestamp;SBname;farmid;tagmac;xval;yval;zval;modul;soft;accX;accY;accZ;absAcc
%20130625182723252;xxxx;182;000008B1;11.01;4.91;;h;5;-71;-583;831;1017.591
%d %s %d %s %f %f %f %s %d %d %d %d %d %f
%read in csv file as txt
[num, txt, raw] = xlsread(fileName); %returns a cell array of strings
%first row is header information read in as string %s
header = textscan(txt{1}, '%s%s%s%s%s%s%s%s%s%s%s%s%s', 'Delimiter', ';'); %13 columns of data... read each of the header titles
header = [header 'time' 'hour' 'minute' 'second' 'ms'];
% loop through remaining rows and extract data (first as text, then as relevant data format)
for iRow=2:length(txt),
%rowData = textscan(txt{iRow}, '%f%s%f%s%f%f%f%s%d%d%d%d%f', 'Delimiter', ';');
rowData = textscan(txt{iRow},'%s%s%s%s%s%s%s%s%s%s%s%s%s','Delimiter', ';'); %just grab as strings, convert later
for iCol=1:length(rowData)
allData(iRow-1,iCol) = rowData{iCol};
end
%change time string to number
allTimes(iRow-1) = allData(iRow-1,1);
%the number to be parsed
a=allTimes';
mstimes=rem(a, 1000000000);
tn=datenum(a,'yyyymmddHHMMSSFFF');
time_hh=str2num(datestr(tn,'HH'));
time_mm=str2num(datestr(tn,'MM'));
time_ss=str2num(datestr(tn,'SS'));
time_ms=str2num(datestr(tn,'FFF'));
end
%add row with time
%allData = [allData times];
cellTime=num2cell(mstimes);
cellHour=num2cell(time_hh);
cellMin=num2cell(time_mm);
cellSec=num2cell(time_ss);
cellMs=num2cell(time_ms);
allData=[allData cellTime cellHour cellMin cellSec cellMs];
allData=sortrows(allData,[3 14]);
output={header; allData};
toc

请先登录，再回答此问题。

回答（3 个）

Image Analyst
Image Analyst 2013-7-16
And the reason you're not using csvread() to read in your csv file is what?????
Seth
Seth 2013-7-17
I'm very bad at Matlab. I just started using it and I'm kind of piecing together what I can from one introductory book. Does csvread() work a lot better?
Don
Don 2018-10-12
编辑：Walter Roberson 2018-10-13
[file, path] = uigetfile('*.csv');
file_name = strcat(path, file);
csv_file = fopen(file_name);
tic;
line = fgetl(csv_file);
% Get the names form the first line
names = textscan(line, '%s');
% this reads 90,002 lines with 20 fields in 4.228 seconds
cntr = 1;
% make a cell to get started
data = cell(10,1);
while ~feof(csv_file)
line2 = fgetl(csv_file);
% Skip the first 2 columns, I don't wnat tme, take this out if you wnat
% them
[DateTime, pos] = textscan(line2, '%s', 2, 'Delimiter', ',');
len = size(line2);
data{cntr} = textscan(line2(pos+1:len(2)), '%f', 'Delimiter', ',');
cntr = cntr+1;
end
fprintf('Read %d lines\n', cntr);
fclose(csv_file);
toc

类别

在 Help Center 和 File Exchange 中查找有关 Text Files 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by