How to read\open a csv file with millions of rows and hundreds of coloumns to compare/delete and save

3 次查看（过去 30 天）

Shayma 2016-7-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/296951-how-to-read-open-a-csv-file-with-millions-of-rows-and-hundreds-of-coloumns-to-compare-delete-and-sav

评论： Shayma 2016-9-22

i have csv files with millions of rows and hundreds of coloumns that i want to open\read in order to compare the files, remove duplicates and save the new file as csv also, and many other modifications..

when i used csvreader the PC stuck! so any help here

9 个评论
显示 7更早的评论隐藏 7更早的评论

Shayma 2016-9-21

编辑：Shayma 2016-9-21

o.k it have been a long time now, but i got the R2014b,i tried to use datastore and it works, at least it opens the first chunk thank you :)

Shayma 2016-9-22

How long it suppose to take reading 100000 lines each chunk from 13.5 GB files ?

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

George 2016-8-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/296951-how-to-read-open-a-csv-file-with-millions-of-rows-and-hundreds-of-coloumns-to-compare-delete-and-sav#answer_231574

在 MATLAB Online 中打开

You can do this with textscan, but your formatSpec is going to be pretty gnarly.

fid = fopen('data.csv');
% your formatSpec will be very long because of the number of fields
formatSpec = '%s %s %f %d';  % reads a string, a string, a float, an integer
A = textscan(fid, formatSpec, 'HeaderLines', 1, 'Delimiter', ',');
fclose(fid);