Slow access of large composite element
1 次查看(过去 30 天)
显示 更早的评论
I am using spmd to run some parallel algorithms which periodically communicate. At the end, each process has a structure which I would like to pull back to the client for further analysis. Unfortunately, the size of the structure is large (but I don't think they are excessive), and trying to pull just one of the composite elements leads to a timeout of the cluster (30 min) and therefore fails. Is there any way to have the client as one of the spmd processes, or a more efficient way to transfer data from one or more of the workers back to the client in a more efficient way?
EDIT:
A clunky workaround is to have each process save their workspace to a directory. This seems to work pretty quickly, but it's obnoxious to have to import all the workspaces to do the diagnostics. I would appreciate more elegant solutions.
5 个评论
Walter Roberson
2018-2-9
Looking more closely, it is for "communicating jobs", of which spmd is one kind.
采纳的回答
更多回答(0 个)
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!