CNN Performance: CPU Consistency vs. GPU Variance - Why?
显示 更早的评论
Hello everyone,
I have executed CNN code multiple times using rng(0) with CPU and consistently obtained the same result. However, when I attempted to accelerate the training process using the GPU, the results differed. Has anyone else faced this issue?
Thank you in advance!
采纳的回答
Ruth
2023-11-23
0 个投票
Hi Hamza,
Even when using "gpurng" some small non-deterministic behavior is expected to happen in the GPU during training, particularly during the backward pass. This is out of our control.
However the behavior should be deterministic in the forward pass and subsequently at prediction time.
If one sets the learning rate to be almost zero (e.g. 1e-16, meaning nothing is updated in the backward pass), the output of training (using "rng" and "gpurng") should look deterministic.
Best wishes,
Ruth
更多回答（1 个）
Edric Ellis
2023-11-23
0 个投票
I'm not certain if it will make everything consistent, but note that random state on the GPU is controlled by the gpurng function.
类别
在 帮助中心 和 File Exchange 中查找有关 Parallel and Cloud 的更多信息
Translated by 
