Fastest Way of Opening and Reading .csv Files (Currently using xlsread)

130 次查看（过去 30 天）
Ibro Tutic
Ibro Tutic 2016-5-23
回答： Kristoffer Walker 2024-8-23
I am currently trying to convert 100,000+ csv files (all the same size, with the same data structuring on the inside) to mat files, and I am running into the issue that it takes an extremely long time, and sometimes Excel stops responding. Are there any other functions that could cut down on the read time of these .csv files?
I read something about trying the COM server that runs Excel, but I am not sure how to implement it. Any thoughts?

请先登录，再回答此问题。

采纳的回答

Kirby Fears
Kirby Fears 2016-5-23
编辑：Kirby Fears 2016-5-23
Thankfully, you don't need to interact with Excel to read csv files. You can use textscan to read csv files quickly.
I wrote a function called delimread that utilizes textscan with automatic parameters. It might be easier than learning how to parameterize textscan.

更多回答（4 个）

Todd Leonhardt
Todd Leonhardt 2016-5-23
Jeremy Hughes
Jeremy Hughes 2017-8-23
Since you have multiple files, you may want to consider using datastore. (Since R2014b)
In many cases, you can just use the following pattern to read a large collection of files,
ds = datastore('folder/containing/your/files')
while(hasdata(ds))
t = read(ds)
% do stuff to t.
end
Hope this helps,
Jeremy
TastyPastry
TastyPastry 2016-5-23
There's a function csvread() which only works on numeric data.
The other way you can do it is to use textscan(). Both of those methods should be faster than xlsread() since xlsread() uses Excel, which is pretty slow.
Kristoffer Walker
Kristoffer Walker 2024-8-23
In my experience, the absolute fastest method is textscan. Here is a benchmark to support my claim using a 3.6 GB CSV file with 4 columns.
Using textscan: 81 seconds
Using readtable: 143 seconds
I tried dlmread and csmread, but they had problems with parsing input. They are not recommended.
Good luck.
Kris

类别

在 Help Center 和 File Exchange 中查找有关 Spreadsheets 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by