save command in matlab parfor loop not going through all iterations
显示 更早的评论
I'm trying to save a matrix within a parfor loop. I know matlab doesn't allow this and one needs to create a separate program. A bit of googling suggest creating a separate function, say parsave as follows:
function parsave(fname, x,y)
save(fname, 'x', 'y')
end
The problem I'm encountering is that the program seems to be halting without any error before going through all of the iterations (i.e., it is saving about half of the files). I've got quite a lot of small files (around 1M). Could this be the problem?
Thanks in advance
采纳的回答
Jason Ross
2012-8-29
0 个投票
You don't provide a lot of detail as to where you are saving the files, but in my experience, when you get above a few thousand files in a directory (dependent on operating system and filesystem type), things start to slow down tremendously. If you are dumping one million files in a single directory, I'd say that you need to re-think that design as the time to do anything with that directory (even something as trivial as a file listing) is going to take forever, even if your OS/filesystem "theoretically" allows it. Your program will be bottlenecked on file I/O, and will be a pain to maintain and troubleshoot over time.
Look into making bigger files or developing a directory structure. Since you are also doing this in parallel and you don't know where the files are going to end up, you need to be very, very careful about duplicate names as well.
Keep in mind that your system is also limited in the number of file handles it can have open, as well, and you could be hitting a limit there. As well as the ability of your hardware to process the file creation requests as the number of files grows.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Parallel Computing Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
