Parallel Computing Question

1 次查看（过去 30 天）
Manoj
Manoj 2012-2-18
Hi, I am a noob in matlab anf I am trying to parallelize loading of an array file.
So suppose the array size is X by Y. I would like to load this file to "n" matlab workers giving X/n by Y to 1st worker and X/n by Y to another and so on.... Does anyone know how can I achieve that since the file is too large and load of that file in a distributed way can save me loads of time.
I tried pmode but since it loads the entire data on each worker and as my data file is huge, it takes a lot of time. So i want to load only X/n by Y on each worker where n is the total number of workers.
So I am trying something like this for my application:
1. Start n worker nodes 2. load X/n by Y data on each worker node 3. Output = function (X/n, Y) 4. Final Output = Gather (Output) 5. Stop all worker nodes.
Do let me know if you can help me in any possible way.
Thanks

请先登录，再回答此问题。

回答（2 个）

the cyclist
the cyclist 2012-2-19
I don't have experience with the parallel computing toolbox, but I speculate that the relatively new matfile() command, which allows loading of partial arrays, maybe be helpful to you.
Jill Reese
Jill Reese 2012-2-21
You might take a look at the DLOAD functionality that is part of the Parallel Computing Toolbox. This function loads distributed arrays and composites from disk, but if you also supply the -scatter option, then DLOAD will also distribute non-distributed data where possible.

类别

在 Help Center 和 File Exchange 中查找有关 Matrix Indexing 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by