Matrix multiplication optimization using GPU parallel computation
显示 更早的评论
Dear all,
I have two questions.
(1) How do I monitor GPU core usage when I am running a simulation? Is there any visual tool to dynamically check GPU core usage?
(2) Mathematically the new and old approaches are same, but why is the new approach is 5-10 times faster?
%%% Code for new approach %%%
M = gpuArray(M) ;
for nt=1:STEPs
if (there is a periodic boundary condition)
M = A1 * M + A2 * f * M
else
% diffusion
M = A1 * M ;
end
end
6 个评论
Jan
2022-8-18
Just curious: What timings do you get for:
M = (A1 + A2 * f) * M;
Are A1, A2 and f gpuArrays also?
Nick
2022-8-18
Nick
2022-8-19
Jan
2022-8-19
Okay. As far as I understand, you do not want to tell me the speed difference between
M = A1 * M + A2 * f * M;
and
M = (A1 + A2 * f) * M
and you do not want to show the complete code for the "old" implementation. Then I cannot estimate, if storing the data in "B(t_n)" is a cause of the problem.
Nick
2022-8-20
Hi Jan,
The following table summarizes the computation time comparison over different approach and GPU enabled/disabled.
New one-step app 1 doesn't have any improvement.
采纳的回答
Because in your second formulation, there is no need to build a table of non-zero entries for the sparse matrix B. The table-building step requires sorting operations, which your second version avoids.
Also, if B has many columns, it will consume a lot of memory in proportion to the number of columns (independent of the sparsity). That is avoided as well by the second implementation.
10 个评论
Nick
2022-8-18
I am surprised that the Matlab compiler didn't optimize this step for the old approach (just substitution as the new approach). Is it due to the MATLAB line-by-line script execution mode?
If you write code instructing Matlab to create a matrix B, Matlab must assume you actually want to use B even if B is never used later in the code. If, for example, you use were to insert a breakpoint in the code, B needs to be available so that you can examine it.
Will MATLAB optimize from old to new if I compile the program into a standaline EXE.
If you convert the code to C/C++ with the Matlab Coder, it might.
Nick
2022-8-22
Matt J
2022-8-22
If you build the matrix on the CPU, then transfer it to the GPU, it would explain why you see CPU activity.
Nick
2022-8-29
There is this rather general doc,
but I want to emphasize that nothing you are seeing is likely related to CPU/GPU transfers or GPU versus CPU differences. It is simply more expensive to create a sparse matrix than to do matrix/vector multiplication with that matrix, even in the plain vanilla case where all processing is done on the CPU (see below). In your case, by avoiding the creation of an additional sparse matrix B, your second version avoids very obvious overhead.
A1=sprand(1e5,1e5,0.001);
A2=sprand(1e5,1e5,0.001);
b=rand(1e5,1);
tic;
B=A1+A2;
toc
tic
B*b;
toc
tic
A1*b; A2*b;
toc
Nick
2022-9-3
Nick
2023-1-19
Do you know how MATLAB manages sparse array elements?
Here is some detail on how sparse matrices are stored,
If so, will any operation on those non-zero elements cause the sorting operations you mentioned above?
If a new sparsity pattern is generated, then it will. Here's maybe another example to show how this can make sparse operations slower than full operations:
N=5000;
A=sprand(N,N,1/5);
B=sprand(N,N,1/5);
tic;
A+B;
toc; %sparse matrix addition
A=full(A); B=full(B);
tic
A+B;
toc %full matrix addition
Nick
2023-1-23
更多回答（1 个）
Joss Knight
2022-8-19
1 个投票
The Windows Task Manager lets you track GPU utilization and memory graphically, and the utility nvidia-smi lets you do it in a terminal window.
Neither the CUDA driver nor the runtime provide access to which core is running what, although you might be able to hand-code something using NVML.
3 个评论
Nick
2022-8-19
Hi Joss,
Thanks for your tip.
It does consume a lot of CPU power (~70%) after executing gpuArray command, and it drops to 12% in the end of GPU simulation.
I observed the dedicated GPU memory usage increase from 0.2 GB to 2.6 GB, but all the GPU performance parameters (e.g. 3D, Copy, Video Encode and Decode) are at almost 0% usage with very tiny ripples.
I am curious what GPU parameter is the key index for matrix multiplication calculation. Would you please advise?
Joss Knight
2022-8-20
Ah, I forgot that you cannot see utilization information for GeForce cards, sorry. Those charts are for graphics and so not relevant for compute (except the memory one).
You'll have to use nvidia-smi.
Nick
2022-8-29
类别
在 帮助中心 和 File Exchange 中查找有关 GPU Computing 的更多信息
Translated by 
