Change the range of potential hyperparameters when optimizing hyperparameter choice using fitrensemble

2020 5 20
1 个回答
回答已采纳
更新时间：2021 7 27
11 次查看（30 天）

1 个投票

Hi,
I am experimenting with randon search and grid search optimizers when optimizing hyperparameters for a boosted trees regression model, using the fitrensemble function.
To speed up the process, and also avoid overfitting, I want to specify non-default ranges to explore for the hyperparamters - however after reading documentation, I still don't understand how to do this.
For example, in the code below:
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'OptimizeHyperparameters',{'NumLearningCycles','LearnRate','MaxNumSplits'},...
'HyperparameterOptimizationOptions',struct(...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1 ));
How would I go about changing the search range for the LearnRate to between 0.001 and 0.2 - rather than the defaul range of [1e-3,1].

请先登录，再回答此问题。

 采纳的回答

Alan Weiss
Alan Weiss 2020-5-20

2 个投票

Looking in the documentation for fitrensemble I find this:
----
Set nondefault parameters by passing a vector of optimizableVariable objects that have nondefault values. For example,
load carsmall
params = hyperparameters('fitrensemble',[Horsepower,Weight],MPG,'Tree');
params(4).Range = [1,20];
Pass params as the value of OptimizeHyperparameters.
----
Is that clear to you? For more information, see https://www.mathworks.com/help/stats/hyperparameters.html#bvdtepr-1
Alan Weiss
MATLAB mathematical toolbox documentation

6 个评论
显示 4更早的评论 隐藏 4更早的评论

Adam Richardson
Adam Richardson 2020-5-21
Thanks!
Therefore, for my question - the approach is (setting range for three hyperparameters and optimizing these):
params = hyperparameters('fitrensemble',X,Y,'Tree');
params(2).Range = [500, 1000] %NumLearningCycles
params(3).Range = [0.001, 0.2] %LearnRate
params(5).Range = [5, 15] %MaxNumSplits
params(1).Optimize = false;
params(2).Optimize = true;
params(3).Optimize = true;
params(4).Optimize = false;
params(5).Optimize = true;
params(6).Optimize = false;
Mdl_ls = fitrensemble(X,Y,'Learners','tree', ...
'OptimizeHyperparameters',params,...
'HyperparameterOptimizationOptions',struct(...
'Optimizer','randomsearch', 'MaxObjectiveEvaluations', 500, 'ShowPlots', false, 'Verbose', 1, 'Holdout', 2/3 ));
Alan Weiss
Alan Weiss 2020-5-21
Good job! Thanks for the follow-up showing your solution.
Alan Weiss
MATLAB mathematical toolbox documentation
@Adam Richardson @Alan Weiss How to restrict the other parameters? For example, I am tyring to use only 'Bag' method in the search space instead of 'LSBoost'
Name: 'Method'
Range: {'Bag' 'LSBoost'}
Type: 'categorical'
Transform: 'none'
Optimize: 1
Alan Weiss
Alan Weiss 2021-7-26
Assuming that the Method hyperparameter is variable 1,
VariableDescriptions(1).Range = {'Bag'};
VariableDescriptions(1).Optimize = false;
Call hyperparameters with your data to find out which variable number it is.
Alan Weiss
MATLAB mathematical toolbox documentation
I am getting following error:
params(1).Range = {'Bag'};
Error using optimizableVariable/checkRange (line 172)
'Range' value must be a string array, a cell array of character vectors, or a numeric vector of length 2.
Alan Weiss
Alan Weiss 2021-7-27
Sorry, my mistake. It seems that if you want to have a single value there you have to repeat it.
params(1).Range = {'Bag' 'Bag'};
params(1).Optimize = false;
Maybe this is bad advice somehow, I don't know, but it seems to work.
Alan Weiss
MATLAB mathematical toolbox documentation

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Get Started with Optimization Toolbox 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by