GPU Coder vs. ONNXRuntime, is there a difference in inference speed?

Question

David 2021-4-1

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/790454-gpu-coder-vs-onnxruntime-is-there-a-difference-in-inference-speed

编辑： Joss Knight 2022-1-7

Since I can export from Matlab to ONNX format, why can't I just import my model into TensorRT etc.? Will I get significant speed increases or is the benefit of GPU Coder more about being able to compile all my other Matlab code into optimized Cuda?

Thanks in advance.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Joss Knight 2021-4-2

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/790454-gpu-coder-vs-onnxruntime-is-there-a-difference-in-inference-speed#answer_664969

You can compile your network for TensorRT using GPU Coder if that's your intended target, no need to go through ONNX.

I don't believe MathWorks have any published benchmarks against ONNX runtime specifically. GPU Coder on the whole outperforms other frameworks, although it does depend on the network.