hyperparameter optimization (deep learning) using bayesopt
3 次查看(过去 30 天)
显示 更早的评论
Following the answer here . I am trying to select best hyperparameters for my Recurrent neural network (RNN).
I want to optimize below hyperparameters in the given code using 'bayesopt()'.
How to define below parameters for 'bayesopt()' using ''optimizableVariable''.
training_function = {'traingd' 'traingda' 'traingdm' 'traingdx'}
optimizers= {'SGD', 'RMSprop', 'Adam'}
activation_functions= {'ReLU','Dropout'};
Transfer_functions= {'tansig,'tanh'};
The complete code is:
% Make some data
Daten = rand(100, 3);
Daten(:,3) = Daten(:,1) + Daten(:,2) + .1*randn(100, 1); % Minimum asymptotic error is .1
[m,n] = size(Daten) ;
% Split into train and test
P = 0.7 ;
Training = Daten(1:round(P*m),:) ;
Testing = Daten(round(P*m)+1:end,:);
XTrain = Training(:,1:n-1);
YTrain = Training(:,n);
XTest = Testing(:,1:n-1);
YTest = Testing(:,n);
% Define a train/validation split to use inside the objective function
cv = cvpartition(numel(YTrain), 'Holdout', 1/3);
% Define hyperparameters to optimize
vars = [optimizableVariable('hiddenLayerSize', [1,20], 'Type', 'integer');
optimizableVariable('epochs', [20,200], 'Type', 'integer')
optimizableVariable('lr', [1e-3 1], 'Transform', 'log')];
----------------------------------
ADD ABOVE HYPERPARAMETERS HERE
--------------------------------
% Optimize
minfn = @(T)kfoldLoss(XTrain', YTrain', cv, T.hiddenLayerSize, T.lr);
results = bayesopt(minfn, vars,'IsObjectiveDeterministic', false,...
'AcquisitionFunctionName', 'expected-improvement-plus');
T = bestPoint(results)
0 个评论
回答(1 个)
Sammit Jain
2020-1-29
Hello Ali,
It appears you're looking to create a BayesianOptimization object, for your set of hyperparameters. The following link has some examples that will help you customize your code:
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!