Matlab and parallel computing
显示 更早的评论
Hello,
I am facing some performance issues while reading data stored inside a wide list of txt files (around 1000000) and each file contains huge matrix data. I do not have PCT and I was thinking to use the java.util.concurrent package for multi-threading stuff (in this case I am looking to the BlockingQueue and ArrayBlockingQueue). Is there any expert that already used this package directly in a .m file? If yes, could you please share it? Do you see any limitation of MATLAB on using this package? Thank you in advance Best regards Paolo
1 个评论
Jan
2017-7-11
You have 1 million text files, which contain numerical data? This would be a really bad method to store the data, because many import and export methods work much more efficient with binary data.
Note that parallel processing does not help, when the single resource of the hard disk's hardware is the bottleneck.
采纳的回答
John D'Errico
2017-7-11
0 个投票
As Jan says, you are probably wasting your time trying to use parallel processing to solve a problem that is not CPU bound, but disk bound.
I'd suggest a better tool is a RAM drive. Speed up the access time instead.
And having them as text files just means that your read time will be terribly slow. That may be something beyond your control.
Finally, I'm not sure this is a good task for MATLAB to solve. If you need the information in MATLAB eventually, then collect it all using some other tool, then read the entire mess to MATLAB in one piece.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Data Import and Export 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
