Deep Learning - Distributed GPU Memory

Question

Daniel Grottke 2020-3-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/512682-deep-learning-distributed-gpu-memory

回答： Joss Knight 2020-3-28

Hello,

I have many, very large input matrices (detector values) connected by Fully Conectet Layer and the output is a Regression Layer to reconstruct an image from it (Only 1 image is used at a time!). Due to the lack of local correlation, a Fully Connected Layer is necessary and CNN cannot be used. But this is beyond the VRAM.

Therefore the question if Matlab with Fully Connected can distribute this?

I have the choice to buy 2x rtx 8000 (2x48GB) or 4x Titan RTX (4x24GB). A RTX 8000 costs 2.5x as much as the Titan RTX and has only the same performance but with twice the memory of a Titan RTX.

2. NV-Link distribute GPU-RAM?

Thanks

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Joss Knight 2020-3-28

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/512682-deep-learning-distributed-gpu-memory#answer_422590

No, there is nothing like what you are after, to distribute the weights of a fully connected layer across multiple GPUs. You could implement it yourself using parallel language constructs, but I assume this is not what you're after.