Failed to generate large CUDA kernel in GPU coder with FFT function inside

Question

He Da 2024-9-18

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2153475-failed-to-generate-large-cuda-kernel-in-gpu-coder-with-fft-function-inside

评论： Justin Hontz 2024-9-19

I am trying to get my code paralle in GPU.

I have converted the code with the "main.m" script as attached. But the mex code on GPU is much slower than the m code on CPU. I understand that the GPU is not suitable for such small data size. But it takes much much longer time on the GPU if bigger data size is used.

Then I check the profilling timeline. I find that many cuda kernel is created and the overall GPU utilization is low. After some debugging, I find that if the fft command is used, the GPU coder failed to generate large CUDA kernel.

I think that the perfermance can be improved significantly if the fft can be incoporate inside one CUDA kernel like the situation without fft. FFT is needed. I have try to search on Google, but nothing relative can be found. Can you provide any information about this or any solution? The output of gpuDevice is also provided in the attachment.

Here is the profilling timeline without fft.

Here is the profilling timeline with fft.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Justin Hontz 2024-9-18

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2153475-failed-to-generate-large-cuda-kernel-in-gpu-coder-with-fft-function-inside#answer_1518955

Hi He,

In your M-code for RandCopy, the for loop cannot be executed as a GPU kernel (even with the coder.gpu.kernel pragma) because of the fft / ifft calls inside of the loop. This is because fft is implemented using its own specialized GPU kernel, and GPU Coder does not supported nested kernels execution. Consequently, the for loop runs sequentially, which explains why you see thousands of small kernel instaces within the performance analyzer timeline graph.

To improve the performance of your code, you will want to perform your computation using only a single fft / ifft call that operates on the entire input array instead of individual slices. Something like this should work:

Tmp = fft(Data,[],2);

Tmp = Tmp + (1 + 1i);

Tmp = Tmp * (1564 + 798i);

Data = ifft(Tmp,[],2);

After making the change on my end, the performance analyzer report shows a significant performance improvement, with the timeline graph looking similar to the original one without fft.

4 个评论
显示 2更早的评论隐藏 2更早的评论

Justin Hontz 2024-9-19

Based on your description of the computation, the best approach would still likely be to rewrite your code similiar to the way that I described above to achieve optimal performance. That is, instead of performing a sequence of operations on each slice of the input array, perform a sequence of larger operations on the entire input array. With this approach, you would still be able to parallelize the computation over the slices. You may end up with multiple kernels being generated instead of just one, but this is unlikely to be a significant performance bottleneck as long as the entire computation can be performed on GPU.

Without seeing your full code, I cannot give any specific advice, though for certain individual operations, you may want to implement them using a for loop with coder.gpu.kernel pragma if they cannot be implemented efficiently using vectorized MATLAB toolbox functions.

Regarding nested kernel execution, combining the code manually is unlikely to work. This is because GPU Coder by default implements fft using the cuFFT API, which is likely not callable from device code. If you still wish to keep your code in its current form, you can also try disabling use of cuFFT from the coder config (see property EnableCUFFT of coder.GpuConfig) and see if that improves the situation.

He Da 2024-9-19

I fully understand the benefit of calculation on the entire array, which is the way I am working for years. However, it is not suitable inherently. I haved tried to disable cuFFT in the coder config, which results thousands of memory copy between the host and device. Maybe it requires other optimzation.

I found something here: https://developer.nvidia.com/cufftdx-downloads

It said:NVIDIA cuFFT introduces cuFFTDx APIs, device side API extensions for performing FFT calculations inside your CUDA kernel. Fusing numerical operations can decrease the latency and improve the performance of your application.

It seems like that the cuFFT can be called from device code. Hopefully you can show me how to use cuFFTDx in RandCopy.m. Perhaps that may be overly demanding.

Justin Hontz 2024-9-19

在 MATLAB Online 中打开

GPU Coder currently does not support generating direct calls to the cuFFTDx API. With that said, however, you may still be able to indirectly call into the API in the generated code if you are willing to write your own CUDA wrapper function that directly uses the API. This can possibly be achieved by invoking the wrapper function inside the for loop of your M-code via coder.ceval. The call would look something like this:

coder.ceval('-gpudevicefcn', 'myFFTWrapper', coder.ref(data), ...);

The -gpudevicefcn flag indicates that the wrapper function is meant to be executed by a GPU thread rather than by the CPU.

Note that I have not tried using this approach on my end, so I cannot guarantee that such an approach would work correctly without issue.

请先登录，再进行评论。

Failed to generate large CUDA kernel in GPU coder with FFT function inside

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

4 个评论
显示 2更早的评论隐藏 2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Failed to generate large CUDA kernel in GPU coder with FFT function inside

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

4 个评论 显示 2更早的评论隐藏 2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

4 个评论
显示 2更早的评论隐藏 2更早的评论