How can I do a simple matrix multiplication withour running in any out of memory error

Question

Mehrnaz Shok 2013-5-18

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/76267-how-can-i-do-a-simple-matrix-multiplication-withour-running-in-any-out-of-memory-error

I want to multiply a big matrix (possibly 6000 * 6000) with its transpose. However I get out of memory error. I know that I don't have enough RAM to make this works...but is there any way I can get this to work?? I am running Matlab R2012b on 32bit windows vista. Can someone show me how to do this block wise, or some methods. The only thing that is important for me is that I need to use the result of the matrix multiplication for another analysis.

This is a sample of what I trying to do: A = ones(6000); B = A'*A; (I need to use B to get another code running).

Please advise, Thanks

2 个评论
显示无隐藏无

per isakson 2013-5-18

编辑：per isakson 2013-5-18

在 MATLAB Online 中打开

R2012a, 64bit, Windows 7, old vanilla desktop, 8GB RAM.

    >> tic, A = ones(6000); B = A'*A; toc
    Elapsed time is 6.767988 seconds.
    >> tic, A = rand(6000); B = A'*A; toc
    Elapsed time is 7.429135 seconds.

I guess from looking at the Task Manager window that 4GB would have been enough so far.

    >> tic, A = rand(9000); B = A'*A; toc
    Elapsed time is 22.454364 seconds.

.

Try

doc memory

Iain 2013-5-22

You can reduce the memory you need by working with singles.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jan 2013-5-18

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/76267-how-can-i-do-a-simple-matrix-multiplication-withour-running-in-any-out-of-memory-error#answer_85917

Perhaps there are any magic tricks, which allow to compute A'*A. When you call the BLAS function through a MEX interface, A' is not calculated explicitly. Perhaps http://www.mathworks.com/matlabcentral/fileexchange/13604-calllapack-matlab-interface-of-lapack-and-blas-functions helps. But nothing can compete with real RAM: Install a 64 bit version and a bunch of GigaBytes. This will be cheaper than about 1 to 2 hours of work with a usual salary for an engineer.

3 个评论
显示 1更早的评论隐藏 1更早的评论

Jan 2013-5-22

@James: I assume this symmetric BLAS routine is triggered by the undocumented JIT acceleartion. Unfortunately I do not know its documentation, such that this could not matter my Matlab version.

Another point could be, that in A = rand(6000); B = A'*A two large matrices must exist at the same time. When A is not required anymore, an inplace construction would be useful. The original BLAS-DGEMM does not exploit, that the result of A'*A is symmetric, as far as I know. Perhaps the MKL and ATLAS implementations do this.

James Tursa 2013-5-22

编辑：James Tursa 2013-5-22

在 MATLAB Online 中打开

@Jan: I am unaware of any specific documentation covering the rules for when symmetric BLAS routines are called. The doc used to list the routines that were used (without giving rules for when they were used), but I don't even see that anymore. E.g., is the symmetric routine called for shared data copy cases such as this:

A = rand(6000);
C = A;
B = C'*A;

Given that the rules are not documented, I don't know if one can always depend on the symmetric behavior.

If you want to know what MATLAB is doing on your particular machine for a particular case, you can always compare the timing to MTIMESX in SPEED mode for a large case. MTIMESX will always key off of the real data pointer value to determine symmetric cases, so it will catch the shared data copy inputs. The timing difference for large symmetric multiplies is an obvious discriminator (something like 30% or more on my machine if I remember correctly). And the symmetric result won't in general match the DGEMM result.

请先登录，再进行评论。