Reading huge number of files quickly (I/O)
3 次查看(过去 30 天)
显示 更早的评论
For analysis of EOD data for stocks, I need to read 3650 .csv files, but using a for loop takes around 30 seconds for 15 files, which would mean around 2 hours just to read data. Is there any way I can do it faster?
0 个评论
回答(1 个)
Walter Roberson
2012-6-4
Are you currently reading using csvread(), or import(), or load() ?
If each file has exactly the same structure (same number of fields, some number of headers, same fields are character vs numeric) then you might be able to do better by using textscan()
If you are currently using xlsread() on MS Windows then you can almost certainly do better than you are doing now (but textscan() might be better still). xlsread() opens and closes Excel each time; Image Analyst has posted code several times showing how to open Excel just once and do several operations with it.
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Financial Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!