Is memory reduction not possible when training a neural network on a GPU?

Question

PetterS 2015-8-3

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/232103-is-memory-reduction-not-possible-when-training-a-neural-network-on-a-gpu

评论： PetterS 2015-8-5

When dealing with large datasets the amount of temporary storage needed during neural network training can be reduced by adding the syntax: “ 'reduction',N” to the training command.

This is a neat feature and would be particularly useful when training nets on graphics cards that generally have less memory compared to the system RAM, but when I add the command to a training done on my GPU the reduction seems to have no effect at all. It doesn’t produce any warning or error message, is simply does not reduce the memory usage of my GPU.

Is this feature not available with GPU training or do I need to do something different in order to make it work?

Thanks.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Nick Hobbs 2015-8-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/232103-is-memory-reduction-not-possible-when-training-a-neural-network-on-a-gpu#answer_188344

The documentation for the train function says that 'reduction' might be able to help with memory only if a MATLAB calculation is being used by the train function. It is possible that the train function is calling a MEX file, and then the reduction likely will not provide much change. This can be determined with 'showResources'. More information on 'showResources' can be found here.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

PetterS 2015-8-5

Yes, I’ve seen that article. But when I do the training on my GPU the resource isn’t Matlab nor MEX, it is simply reported as “CUDA”. I don’t know if CUDA calculations take place in Matlab or in MEX or if it counts as neither.

请先登录，再进行评论。