Eigs in multinode cluster

8 次查看（过去 30 天）
Wister Huang
Wister Huang 2016-5-18
编辑： Andrew Knyazev 2019-6-12
I am new to high performance computing and I am now trying to reconfigure one of my matlab code to a HPC.(I know it is probably not preferred, but I kind of need it in urgent.)
The task is to solve the lowest 6 eigen values for a 42500*42500 complex sparse Hermician matrix.
I have access to minimum 4 nodes and each node have more than 16GB memory.
I tried to submit this job but get an error
Out of memory. Type HELP MEMORY for your options.
Error in eigs/checkInputs/LUfactorAminusSigmaB (line 987)
[L,U,pp] = lu(AsB,'vector');
Error in eigs/checkInputs (line 809)
[L,U,pp,qq,dgAsB] = LUfactorAminusSigmaB;
Error in eigs (line 96)
checkInputs(varargin{:});
Error in TwoDsolve (line 65)
[psi,E]=eigs(H,NOS,'sm');
Error in Loop_Main (line 9)
[psi,En] = TwoDsolve( V,m2Dx,m2Dy,Grd,'eigs',NOS);
Apparently it only use My question is: is there a way to configure eigs so that it can use all the memory I required from different nodes? I attach my submission file here as well.
#SBATCH --mail-user=wister.huang@student.unsw.edu.au
#SBATCH --mail-type=ALL
#SBATCH --time=00:60:00
#SBATCH --nodes=4
#SBATCH --mem=16384
#SBATCH --ntasks=4
#SBATCH --cpus-per-task=4
echo "myjob.sub"
hostname
module load matlab/2016a
cd $PBS_O_WORKDIR
unset DISPLAY
matlab -nodisplay -r Control
Thanks, Wister

请先登录，再回答此问题。

采纳的回答

Andrew Knyazev
Andrew Knyazev 2018-8-11
编辑：Andrew Knyazev 2019-6-12
EIGS has limited support for distributed memory, so you can run it only on a single node, but see the answer from Christine Tobler below.
Google, e.g., for LOBPCG implementations in BLOPEX, SLEPc, Trilinos, hypre, MAGMA for parallel MPI-based eigensolvers that scale to thousands of nodes and run on GPUs.
https://www.mathworks.com/matlabcentral/fileexchange/48-locally-optimal-block-preconditioned-conjugate-gradient
Revision 4.16 adds support for distributed or codistributed arrays available in MATLAB BigData toolbox, e.g.,:
A = codistributed(diag(1:100)); B = codistributed(diag(101:200));
[blockVectorX,lambda]=lobpcg(randn(100,2),A,1e-5,5,2)

更多回答（1 个）

Christine Tobler
Christine Tobler 2018-8-13
The eigs function is supported for codistributed matrices in the Parallel Toolbox (see help codistributed/eigs. However, this version of eigs does not support the 'sm' / 'smallestabs' option (except for the case of a diagonal or triangular input matrix).
You could try to use the 'smallestreal' option instead, but this often has convergence issues, as it does not compute a decomposition of A, and also doesn't use a preconditioner, as LOBPCG typically does. However, this should be quick enough to try out, as it is just a one-liner.
  1 个评论
Christine Tobler
Christine Tobler 2018-8-13
Sorry, I realize I started out with the details when I should have started more generally: To distribute a MATLAB array over several compute nodes, use the Parallel Toolbox, and its distributed or codistributed classes (<https://www.mathworks.com/products/parallel-computing.html Parallel Toolbox Documentation>).

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Distributed Arrays 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by