Matlab and parallel computing

1 次查看（过去 30 天）
Paolo
Paolo 2017-7-11
回答： John D'Errico 2017-7-11
Hello,
I am facing some performance issues while reading data stored inside a wide list of txt files (around 1000000) and each file contains huge matrix data. I do not have PCT and I was thinking to use the java.util.concurrent package for multi-threading stuff (in this case I am looking to the BlockingQueue and ArrayBlockingQueue). Is there any expert that already used this package directly in a .m file? If yes, could you please share it? Do you see any limitation of MATLAB on using this package? Thank you in advance Best regards Paolo
  1 个评论
Jan
Jan 2017-7-11
You have 1 million text files, which contain numerical data? This would be a really bad method to store the data, because many import and export methods work much more efficient with binary data.
Note that parallel processing does not help, when the single resource of the hard disk's hardware is the bottleneck.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

John D'Errico
John D'Errico 2017-7-11
As Jan says, you are probably wasting your time trying to use parallel processing to solve a problem that is not CPU bound, but disk bound.
I'd suggest a better tool is a RAM drive. Speed up the access time instead.
And having them as text files just means that your read time will be terribly slow. That may be something beyond your control.
Finally, I'm not sure this is a good task for MATLAB to solve. If you need the information in MATLAB eventually, then collect it all using some other tool, then read the entire mess to MATLAB in one piece.

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Startup and Shutdown 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by