TreeBagger parameter tuning for classification

Question

kawther hassine 2018-6-7

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/404625-treebagger-parameter-tuning-for-classification

评论： Marta Caneda Portela 2022-9-6

How can I tune parameters for TreeBagger model for classification, I followed the example:"Tune Random Forest Using Quantile Error and Bayesian Optimization", https://fr.mathworks.com/help/stats/tune-random-forest-using-quantile-error-and-bayesian-optimization.html I only changed "regression" with "classification". The following code generated multiple errors:

results = bayesopt(@(params)oobErrRF(params,X),hyperparametersRF,...
    'AcquisitionFunctionName','expected-improvement-plus','Verbose',0);

errors:

Error using classreg.learning.internal.table2FitMatrix>resolveName (line 232)
One or more 'ResponseName' parameter values are invalid.
Error in classreg.learning.internal.table2FitMatrix (line 77)
        ResponseName = resolveName('ResponseName',ResponseName,FormulaResponseName,false,VarNames);
Error in ClassificationTree.prepareData (line 557)
            [X,Y,vrange,wastable,varargin] =
            classreg.learning.internal.table2FitMatrix(X,Y,varargin{:},'OrdinalIsCategorical',false);
Error in TreeBagger/init (line 1335)
                    ClassificationTree.prepareData(x,y,...
Error in TreeBagger (line 615)
            bagger = init(bagger,X,Y,makeArgs{:});
Error in oobErrRF2 (line 16)
randomForest = TreeBagger(300,X,'MPG','Method','classification',...
Error in @(params)oobErrRF2(params,trainingDataFeatures)
Error in BayesianOptimization/callObjNormally (line 2184)
                        Objective = this.ObjectiveFcn(conditionalizeX(this, X));
Error in BayesianOptimization/callObjFcn (line 2145)
                    = callObjNormally(this, X);
Error in BayesianOptimization/callObjFcn (line 2162)
                            = callObjFcn(this, X);
Error in BayesianOptimization/performFcnEval (line 2128)
                ObjectiveFcnObjectiveEvaluationTime, this] = callObjFcn(this, this.XNext);
Error in BayesianOptimization/run (line 1836)
                this = performFcnEval(this);
Error in BayesianOptimization (line 450)
            this = run(this);
Error in bayesopt (line 287)
Results = BayesianOptimization(Options);

I would like to know if there is a way to use this method of tuning for classification. If not, how can I tune my parameters for a TreeBagger classifier. Thanks.

2 个评论
显示无隐藏无

Don Mathis 2018-6-8

What version of MATLAB are you using? That's not the error I get using R2018a

kawther hassine 2018-6-18

It's R2017a.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Don Mathis 2018-6-8

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/404625-treebagger-parameter-tuning-for-classification#answer_323812

在 MATLAB Online 中打开

The following works for me in R2018a. It predicts 'Cylinders' (3 classes) and it calls oobError to get the misclassification rate of the ensemble.

load carsmall
Cylinders = categorical(Cylinders);
Mfg = categorical(cellstr(Mfg));
Model_Year = categorical(Model_Year);
X = table(Acceleration,Cylinders,Displacement,Horsepower,Mfg,...
    Model_Year,Weight,MPG);
rng('default'); % For reproducibility
maxMinLS = 20;
minLS = optimizableVariable('minLS',[1,maxMinLS],'Type','integer');
numPTS = optimizableVariable('numPTS',[1,size(X,2)-1],'Type','integer');
hyperparametersRF = [minLS; numPTS];
results = bayesopt(@(params)oobErrRF(params,X),hyperparametersRF,...
    'AcquisitionFunctionName','expected-improvement-plus','Verbose',1);
bestOOBErr = results.MinObjective
bestHyperparameters = results.XAtMinObjective
Mdl = TreeBagger(300,X,'Cylinders','Method','classification',...
    'MinLeafSize',bestHyperparameters.minLS,...
    'NumPredictorstoSample',bestHyperparameters.numPTS);
function oobErr = oobErrRF(params,X)
%oobErrRF Trains random forest and estimates out-of-bag quantile error
%   oobErr trains a random forest of 300 regression trees using the
%   predictor data in X and the parameter specification in params, and then
%   returns the out-of-bag quantile error based on the median. X is a table
%   and params is an array of OptimizableVariable objects corresponding to
%   the minimum leaf size and number of predictors to sample at each node.
randomForest = TreeBagger(300,X,'Cylinders','Method','classification',...
    'OOBPrediction','on','MinLeafSize',params.minLS,...
    'NumPredictorstoSample',params.numPTS);
oobErr = oobError(randomForest, 'Mode','ensemble');
end

9 个评论
显示 7更早的评论隐藏 7更早的评论

kawther hassine 2018-6-18

在 MATLAB Online 中打开

Thanks Don Mathis for your response. This example works, however when I wanted to apply the same optimization on my data it doesn't. It generates multiple errors:

 Warning: Variable names were modified to make them valid MATLAB identifiers. The original names are saved in the
VariableDescriptions property. 
Warning: Variable names were modified to make them valid MATLAB identifiers. The original names are saved in the
VariableDescriptions property. 
Error using classreg.learning.internal.table2FitMatrix>resolveName (line 232)
One or more 'ResponseName' parameter values are invalid.

Error in classreg.learning.internal.table2FitMatrix (line 77) ResponseName = resolveName('ResponseName',ResponseName,FormulaResponseName,false,VarNames);

Error in ClassificationTree.prepareData (line 557) [X,Y,vrange,wastable,varargin] = classreg.learning.internal.table2FitMatrix(X,Y,varargin{:},'OrdinalIsCategorical',false);

Error in TreeBagger/init (line 1335) ClassificationTree.prepareData(x,y,...

Error in TreeBagger (line 615) bagger = init(bagger,X,Y,makeArgs{:});

Error in ModelTuning2>oobErrRF (line 61) randomForest = TreeBagger(21,X,'trainedDataAttackLabel','Method','classification',...

Error in ModelTuning2>@(params)oobErrRF(params,X)

Error in BayesianOptimization/callObjNormally (line 2184) Objective = this.ObjectiveFcn(conditionalizeX(this, X));

Error in BayesianOptimization/callObjFcn (line 2145) = callObjNormally(this, X);

Error in BayesianOptimization/callObjFcn (line 2162) = callObjFcn(this, X);

Error in BayesianOptimization/performFcnEval (line 2128) ObjectiveFcnObjectiveEvaluationTime, this] = callObjFcn(this, this.XNext);

Error in BayesianOptimization/run (line 1836) this = performFcnEval(this);

Error in BayesianOptimization (line 450) this = run(this);

Error in bayesopt (line 287) Results = BayesianOptimization(Options);

Error in ModelTuning2 (line 46) results = bayesopt(@(params)oobErrRF(params,X),hyperparametersRF,...

kawther hassine 2018-6-19

在 MATLAB Online 中打开

I am not sure, I have been working with these variable names without errors (in other programs). Now I am getting different errors. This is the code I used:

load networkTraffic
proto = categorical(cellstr(proto));
service = categorical(cellstr(service));
state = categorical(cellstr(state));
attack_cat=categorical(attack_cat); % Response
X = table(dur,proto,service,state,spkts,dpkts,sbytes,dbytes,rate,sttl,dttl,sload,dload,sloss,dloss,sinpkt,dinpkt,sjit,djit,swin,stcpb,dtcpb,dwin,...
    tcprtt,synack,ackdat,smean,dmean,trans_depth,response_body_len,ct_srv_src,ct_state_ttl,...
    ct_dst_ltm,ct_src_dport_ltm,ct_dst_sport_ltm,ct_dst_src_ltm,is_ftp_login,ct_ftp_cmd,...
    ct_flw_http_mthd,ct_src_ltm,ct_srv_dst,is_sm_ips_ports);

At this point, Matlab does not recognize the variable dbytes, wich is is strange because it seems to recognize the ones before dbytes. The error is:

   Warning: Variable names were modified to make them valid MATLAB identifiers. The original names are saved
  in the VariableDescriptions property. 
  Warning: Variable names were modified to make them valid MATLAB identifiers. The original names are saved
  in the VariableDescriptions property. 
  Undefined function or variable 'sbytes'.
Error in ModelTuning2 (line 34)
X =
table(dur,proto,service,state,spkts,dpkts,sbytes,dbytes,rate,sttl,dttl,sload,dload,sloss,dloss,sinpkt,dinpkt,sjit,djit,swin,stcpb,dtcpb,dwin,...

Thanks.

kawther hassine 2018-6-25

在 MATLAB Online 中打开

Here you find the complete code. You could find the data set at NetworkTraffic. I hope you could find the error. Thanks.

load networkTraffic.mat
proto= categorical(cellstr(proto));
service= categorical(cellstr(service));
state = categorical(cellstr(state));
attackCat=categorical(cellstr(attack_cat));
X = table(dur,proto,service,state,spkts,dpkts,sbytes,dbytes,rate,sttl,dttl,sload,dload,sloss,...
    dloss,sinpkt,dinpkt,sjit,djit,swin,stcpb,dtcpb,dwin,tcprtt,synack,ackdat,smean,dmean,trans_depth,response_body_len,ct_srv_src,ct_state_ttl,...
    ct_dst_ltm,ct_src_dport_ltm,ct_dst_sport_ltm,ct_dst_src_ltm,is_ftp_login,ct_ftp_cmd,...
    ct_flw_http_mthd,ct_src_ltm,ct_srv_dst,is_sm_ips_ports);
rng('default'); % For reproducibility
maxMinLS = 20;
minLS = optimizableVariable('minLS',[1,maxMinLS],'Type','integer');
numPTS = optimizableVariable('numPTS',[1,size(X,2)-1],'Type','integer');
hyperparametersRF = [minLS; numPTS];
results = bayesopt(@(params)oobErrRF(params,X),hyperparametersRF,...
    'AcquisitionFunctionName','expected-improvement-plus','Verbose',1);
bestOOBErr = results.MinObjective
bestHyperparameters = results.XAtMinObjective
Mdl = TreeBagger(100,X,'attackCat','Method','classification',...
    'MinLeafSize',bestHyperparameters.minLS,...
    'NumPredictorstoSample',bestHyperparameters.numPTS);
function oobErr = oobErrRF(params,X)
%oobErrRF Trains random forest and estimates out-of-bag quantile error
%   oobErr trains a random forest of 300 regression trees using the
%   predictor data in X and the parameter specification in params, and then
%   returns the out-of-bag quantile error based on the median. X is a table
%   and params is an array of OptimizableVariable objects corresponding to
%   the minimum leaf size and number of predictors to sample at each node.
randomForest = TreeBagger(100,X,'attackCat','Method','classification',...
    'OOBPrediction','on','MinLeafSize',params.minLS,...
    'NumPredictorstoSample',params.numPTS);
oobErr = oobError(randomForest, 'Mode','ensemble');
end

Don Mathis 2018-6-26

在 MATLAB Online 中打开

>> load networkTraffic.mat
>> proto= categorical(cellstr(proto));
Undefined function or variable 'proto'.

Marta Caneda Portela 2022-9-6

What if we need to do kFold validation to optimize hyperparameters?

请先登录，再进行评论。

TreeBagger parameter tuning for classification

2 个评论
显示无隐藏无

回答（1 个）

9 个评论
显示 7更早的评论隐藏 7更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

TreeBagger parameter tuning for classification

2 个评论 显示 无隐藏 无

回答（1 个）

9 个评论 显示 7更早的评论隐藏 7更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

WeChat

2 个评论
显示无隐藏无

9 个评论
显示 7更早的评论隐藏 7更早的评论