Efficient way to read in large dataset

1 次查看（过去 30 天）
Mads
Mads 2011-11-1
Hi all.
I am to read in a set of data to Matlab which has 6 columns and more than 2 million rows. I tried reading the data as a tab delimited .txt file with the dataset function as such:
pricedata = ...
dataset('File',file,'Delimiter','tab','ReadObsNames',true) ;
This has gone on for more than 3 hours. Can someone suggest a better way to do it?

请先登录，再回答此问题。

采纳的回答

Peter Perkins
Peter Perkins 2011-11-1
Mads, you will most likely find that this:
>> help dataset/dataset
dataset Create a dataset array.
[snip]
DS = dataset('File',FILENAME,'Format',FORMAT, ...) creates
a dataset array using the TEXTSCAN function to read column-
oriented data in a text file. Specifying a format can
improve speed significantly for large files.
will read the file much faster. In your case, FORMAT might be something like '%f%f%f%f%f%f' if all the columns are numeric. Hope it helps.

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Data Type Conversion 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by