GPUを使用したDeepLearningの並列処理に関する質問

Question

Tomohiro Nagai 2017-7-6

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347584-gpu-deeplearning

评论： michio 2017-7-12

現在，GPU（GeForce 1080-Ti）を使用したCNNによる解析を行っています．1枚のGPUで動作させていますが，高速処理のため，複数枚数（2枚以上）のGPUを考えています．どのような仕様（また，環境下）であれば，CNNの複数GPUを用いた処理が可能でしょうか．以下の環境下で動かしたいと考えています．

・Windows10環境下・GPU（GeForce 1080-Ti）×2枚・Parallel Computing Toolboxは利用可能

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

michio 2017-7-7

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347584-gpu-deeplearning#answer_273170

具体的な実行方法としては下記が参考になるかと思います。

4 个评论
显示 2更早的评论隐藏 2更早的评论

Joss Knight 2017-7-11

Many apologies, I should have noticed they were links! (They weren't links in Google translate of course...)

michio 2017-7-12

No worries :)

请先登录，再进行评论。

Answer 2

Joss Knight 2017-7-6

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347584-gpu-deeplearning#answer_273161

编辑：Joss Knight 2017-7-6

MATLAB CNNs support any modern multi-gpu environment and it's difficult to advise. Amazon and Azure provide cloud environments of 8 and 16 Tesla K80 GPUs, while the typical 'multi-gpu system in a box' retail system would allow you to put together 4 or 8 GeForce GPUs (1080 or Titan X) and that would be equally useful for Deep Learning. If you just want to try it out, put another 1080i in your desktop, if your PCI bus can take another.

There are two bits of advice I can usefully give. Firstly, avoid setting up your multi-gpu cluster under Windows, because MATLAB's cluster model is multi-process not multi-threaded. Multi-process scales better to multiple machines, but it prevents peer-to-peer communication between GPUs on Windows, since P2P inter-process communication in CUDA is only supported on Linux (for now). It's worth pointing out that you can still use MATLAB under Windows as your client while the cluster runs under Linux.

Secondly, try to put as many GPUs as possible on the PCI bus of one machine, rather than dividing them between multiple machines. Again, this is because of peer-to-peer communication, which is essential for fast training of deep networks. Communicating between nodes on a multi-node cluster is much slower than peer communication between GPUs on the same PCI bus.