Out of memory smpd parallel computing toolbox eig on complex matrix

2 次查看（过去 30 天）
Renee
Renee 2017-2-2
评论： Renee 2017-2-8
Hi, I have an 11x11 matrix from a system of 11 ODEs (hence the complexity). Matlab's eig was unable to solve the matrix without running out of memory, so I'm trying out the parallel computing toolbox. I haven't been able to find any clear instructions, so I may be doing very obvious things wrong. My code is:
parpool
spmd
tic
[l1,l2] = eig(D);
time = gop(@max, toc) % Time for all labs in the pool to complete.
end
delete(gcp)
and D is my matrix. I set up my local profile so that I use 6 workers. My question is: am I using the toolbox/functions incorrectly, or is my system just too complex to solve? In case relevant, I am running Matlab 2016b on win10 64, 4 ghz cpu, 24 gb ram. It gets up to almost 100% memory usage and about 80% cpu usage with this code.
The error:
Starting parallel pool (parpool) using the 'local' profile ... connected to 6 workers.
ans =
Pool with properties:
Connected: true
NumWorkers: 6
Cluster: local
AttachedFiles: {}
IdleTimeout: 30 minute(s) (30 minutes remaining)
SpmdEnabled: true
Warning: An error has occurred during SPMD execution. An attempt has been made to interrupt execution on the workers. If this situation persists, it may be necessary to interrupt execution using CTRL-C and then deleting and
restarting the parallel pool.
The error that occurred on worker 3 is:
Error using symengine
Out of memory.
Error in sym/mupadmexnout (line 981)
out = mupadmex(fcn,args{:});
Error in sym/eig (line 54)
[Vsym,Dsym,p] = mupadmexnout('symobj::eigenvectors',A);
Error in eigen_test1 (line 50)
spmd
.
> In spmdlang.RemoteSpmdExecutor/maybeWarnIfInterruptedAndWaiting (line 300)
In spmdlang.RemoteSpmdExecutor/isComputationComplete (line 131)
In spmdlang.spmd_feval_impl (line 19)
In spmd_feval (line 8)
In eigen_test1 (line 50)
Error using eigen_test1 (line 50)
Error detected on worker 3.
Caused by:
Error using symengine
Out of memory.
Exception in thread "Explorer NavigationContext request queue" IdleTimeout has been reached.
Parallel pool using the 'local' profile is shutting down.
  1 个评论
Edric Ellis
Edric Ellis 2017-2-3
In this case, unfortunately I think you are simply replicating the same computation on each worker. Inside an spmd block, to get parallelism, you need to use codistributed arrays. But it looks here like you are using symbolic computation.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Prannay Jain
Prannay Jain 2017-2-7
I understand that you are running into memory issues even after parallelizing the computation using spmd and parpool.
Moving to Parallel Computing Toolbox (PCT) while remaining on the same machine will never resolve a memory issue. By splitting the computation across multiple workers makes the memory situation worse by having extra MATLABs open. You will either need a bigger machine with more memory or try considering out a cluster where you can appropriately distribute the problem across multiple machines' memory using MATLAB Distributed Computing Server (MDCS) and PCT.

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Parallel Computing Fundamentals 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by