Read big file with mixed data types with datastore

23 次查看（过去 30 天）
Sy Dat Ho
Sy Dat Ho 2024-11-25，13:30
评论： Sy Dat Ho 30 minutes 前
I've got a file which is 300 GB big. A piece of it can be found in the attached file. I've read that the best way to handle this kind of files is to read them into a datastore.
As you can see, the first two lines are characters, while the following lines are a combination of floats and integers. Is it possible to read them predefined? I know from fscanf that you can specify the data type, but when I do datastore it interprets every line as a string.

请先登录，再回答此问题。

回答（1 个）

Stephen23
Stephen23 2024-11-25，14:05
Ran in:
ds = datastore('./*.txt', 'Type','tabulartext', 'NumHeaderLines',2, 'TextscanFormats',repmat("%f",1,5));
T = preview(ds)
T = 8x5 table
Var1 Var2 Var3 Var4 Var5 ____ _______ _______ _______ ______ 192 0 0 0 NaN 108 0.21721 0 0 NaN 108 0 0.21721 0 NaN 108 0 0 0.21721 NaN 8 0 17.09 2.3461 1.2766 8 0 21.968 21.103 17.839 8 0 14.849 17.511 11.303 8 0 22.723 23.318 13.066
  3 个评论
显示 1更早的评论
Stephen23
Stephen23 2024-11-25，14:24
Ran in:
fid = fopen('test.txt','rt');
hd1 = fgetl(fid)
hd1 = '-Quickstep-'
hd2 = fgetl(fid)
hd2 = 'SPIN DENSITY'
fclose(fid);
Sy Dat Ho
Sy Dat Ho 30 minutes 前
Thanks but i can't use fopen bc my ram is smaller than the file.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Datastore 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by