Double precision in deep learning
显示 更早的评论
Hello guys,
i want to train my LSTM using double precision but the deep learning toolbox only allows single precision. Can i change that to double precision or how do i train my neural network using double precision ?
Thanks.
5 个评论
KSSV
2022-11-28
To my knowledge, the present deep learning toolbox uses single precision. Though you give input in double, it will be converted to single and trained.
bah
2022-11-29
Matt J
2022-11-29
I don't know, but you should be mindful that training could be slower on the GPU, not just because of the increased precision, but because some GPUs are not optimized for double precision.
bah
2022-11-29
Walter Roberson
2023-11-12
A lot of Nvidia models have double precision at 1/32 of single precision rates. Some improve to 1/24. And a few uncommon models more intended for server improve to 1/8: you have to examine the models carefully to determine the best for your purposes.
采纳的回答
Joss Knight
2022-12-18
This is possible with a dlnetwork but it is (currently) more of a workaround than anything else. Once your dlnetwork is ready to go, convert it to double precision:
net = dlupdate(@double, net);
net.State = dlupdate(@double, net.State);
Then make sure your input data to the network is always in double precision. Everything should then remain in double as you train and use the network.
Let me know if this doesn't work for you.
2 个评论
Rohan Mahala
2023-11-12
Hi Joss, can you please tell how to do this without a dlnetwork
Joss Knight
2023-11-12
To do this without a dlnetwork you will need to implement your network in functional form. Either way, to train it will require use of automatic differentiation and a custom training loop.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
