Working with LSTM and Bayes Optimization

Question

CHRISTOPHER MILLAR 2020-2-25

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/507418-working-with-lstm-and-bayes-optimization

评论： CHRISTOPHER MILLAR 2021-10-5

I am trying to use bayesoptimization to tune the parameters

optimvars = [
    optimizableVariable('InitialLearnRate',[1e-2 1],'Transform','log')
    optimizableVariable('L2Regularization',[1e-10 1e-2],'Transform','log')];
layers = [ ...
    sequenceInputLayer(inputSize,'Normalization','zscore')
    bilstmLayer(numHiddenUnits,'OutputMode','last')
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer];
maxEpochs =25;
options = trainingOptions('adam',...
    'ExecutionEnvironment','cpu',...
    'GradientThreshold',1,...
    'MaxEpochs',maxEpochs,...
    'MiniBatchSize',miniBatchSize, ...
    'SequenceLength', 'longest', ...
    'Shuffle','every-epoch', ...
    'Verbose', 1, ...
    'InitialLearnRate',optimvars.InitialLearnRate,...
    'L2Regularization',optimvars.L2Regularization,...
    'Plots','training-progress');
objFcn = makeObj(Xtrain,YTrain);
bayesObj = bayesopt(objFcn,optimvars, ...
    'MaxTime', 14*60*60, ...
    'IsObjectiveDeterministic',false,...
    'UseParallel',false);

Where am i going wrong as i get the following error:

Unrecognized method, property, or field 'InitialLearnRate' for class 'optimizableVariable'.
Error in AllVsIndx (line 236)
    'InitialLearnRate',optimvars.InitialLearnRate,...

The documentation regarding bayesian optimization is very vague especially when it comes to implementation with LSTM networks

Any help would be appreciated

Thanks

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jorge Calvo 2021-5-27

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/507418-working-with-lstm-and-bayes-optimization#answer_710550

If you have R2020b or later, you can use the Experiment Manager app to run Bayesian optimization to determine the best combination of hyperparameters. For more information, see https://www.mathworks.com/help/deeplearning/ug/experiment-using-bayesian-optimization.html.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

CHRISTOPHER MILLAR 2021-5-27

Thanks Jorge

I have just installed R2021a on my machine and will take advantage of this new app

请先登录，再进行评论。

Answer 2

Don Mathis 2020-2-25

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/507418-working-with-lstm-and-bayes-optimization#answer_417312

You might find this similar example useful: https://www.mathworks.com/matlabcentral/answers/457788-lstm-time-series-hyperparameter-optimization-using-bayesian-optimization?s_tid=answers_rc1-2_p2_MLT

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 3

Jorge Calvo 2021-10-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/507418-working-with-lstm-and-bayes-optimization#answer_801886

I thought you would like to know that, in R2021b, we are included an example for training long short-term memory (LSTM) networks using Bayesian optimization in Experiment Manager:

https://www.mathworks.com/help/deeplearning/ug/exp-mgr-sequence-regression-example.html

I hope you find it helpful!