GPU computing for machine learning (bagging / ensemble)

11 次查看（过去 30 天）
Joe
Joe 2015-8-20
评论： Ilya 2015-8-25
Hi, Is bagging / ensemble supported by gpu computing in matlab?
I need to create a random forest that requires lots of processing time and was wondering if I can accelerate issuing gpu computing.
Thanks

请先登录，再回答此问题。

回答（4 个）

Chetan Rawal
Chetan Rawal 2015-8-21
Yes, treebagger is supported and has in-built GPU support. Give it a try: http://www.mathworks.com/products/parallel-computing/builtin-parallel-support.html
You should be able to grow the trees on the GPU. Then also might gain further performance by aggregating your ensemble on multiple cores of the CPU using Parallel Computing Toolbox. I'd suggest profiling your code first to see if this second step will help.
Chetan
Ilya
Ilya 2015-8-21
There is no GPU support for decision trees or their ensembles. If you work in a sufficiently recent release, decision trees are multithreaded. In addition, TreeBagger, as noted, has parallel support through Parallel Computing Toolbox.
Can you tell us about your data size (number of observations and predictors) and time requirements? Have you already tried fitting a random forest and concluded it is too slow for your case?
Joe
Joe 2015-8-22
Hi,
Thanks for the comments. I am already using parallel computing with my 6 cores.
I have around 10 million observations, around 100 predictors, and very weak signals. I fitted a tree and took 20 mins. However this is one piece of a big optimization and need to do a few thousands.
Any help appreciated!
  1 个评论
Ilya
Ilya 2015-8-23
编辑：Ilya 2015-8-23
Try boosting. You don't say much about your data, so I can't recommend a specific boosting algorithm. Use at least a few dozen trees and play with the minimal leaf size ('minleaf') to obtain the optimal accuracy on an independent test set.
If you insist on using TreeBagger, likewise play with the minimal leaf size. By default, trees for random forest are split to a very fine level (minleaf=1 for classification and 5 for regression). You likely don't need such deep trees for 10M observations. Increasing the leaf size would speed up training and reduce the memory footprint of the ensemble a lot.
Boosting typically outperforms random forest in accuracy on large datasets. Boosting trees is also faster because you can keep trees fairly shallow. The disadvantage is that you need to spend more time searching for optimal values of the boosting parameters such as the minimal leaf size and something else (say learning rate for some algorithms).
If you are using 15a or later, trees are multithreaded. parfor across local cores is not going to help you much and can even lead to a slowdown because you would be consuming quite a bit more memory.

请先登录，再进行评论。

Joe
Joe 2015-8-24
Thanks a lot Ilya!
What are the things about the data you need to know to be able to provide a recommendation on specific boosting algorithm?

类别

在 Help Center 和 File Exchange 中查找有关 Parallel and Cloud 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by