LSTM time series hyperparameter optimization using bayesian optimization

28 次查看（过去 30 天）
anurag kulshrestha
anurag kulshrestha 2019-4-22
回答： SIVA SRI 2024-9-3
I am working with time series regression problem. I want to optimize the hyperparamters of LSTM using bayesian optimization. I have 3 input variables and 1 output variable.
I want to optimize the number of hidden layers, number of hidden units, mini batch size, L2 regularization and initial learning rate . Code is given below:
numFeatures = 3;
numHiddenUnits = 120;
numResponses = 1;
layers = [ ...
sequenceInputLayer(numFeatures)
lstmLayer(numHiddenUnits,'OutputMode','sequence')
fullyConnectedLayer(numResponses)
regressionLayer];
options = trainingOptions('adam', ...
'MaxEpochs',maxEpochs, ...
'MiniBatchSize',miniBatchSize, ...
'InitialLearnRate',optVars.InitialLearnRate, ...
'Momentum',optVars.Momentum, ...
'GradientThreshold',1, ...
'Shuffle','never', ...
'L2Regularization',optVars.L2Regularization, ...
'Plots','training-progress',...
'Verbose',0);
net = trainNetwork(XTrain,YTrain,layers,options);
YPredicted = predict(net,Xval, 'MiniBatchSize',1);
valError = 1 - mean(YPredicted == Yval);
Thanks in advance.

请先登录，再回答此问题。

回答（4 个）

Jorge Calvo
Jorge Calvo 2021-10-5
I thought you would like to know that, in R2021b, we are included an example for training long short-term memory (LSTM) networks using Bayesian optimization in Experiment Manager:
https://www.mathworks.com/help/deeplearning/ug/exp-mgr-sequence-regression-example.html
I hope you find it helpful!
Don Mathis
Don Mathis 2019-5-10
Here's an example using a convolutional network instead of an LSTM network. Your LSTM case should look very similar: https://www.mathworks.com/help/deeplearning/examples/deep-learning-using-bayesian-optimization.html
  1 个评论
Sinan Islam
Sinan Islam 2021-5-8
LSTM is different from CNN. It is obvious that this example is in great demand. Why not Matlab make a proper example dedicated for optimizing LSTM?

请先登录，再进行评论。

Jorge Calvo
Jorge Calvo 2021-5-27
If you have R2020b or later, you can use the Experiment Manager app to run Bayesian optimization to determine the best combination of hyperparameters. For more information, see https://www.mathworks.com/help/deeplearning/ug/experiment-using-bayesian-optimization.html.
  2 个评论
Sinan Islam
Sinan Islam 2021-5-27
编辑：Sinan Islam 2021-5-27
Can I use Experiment Manager to load 200 different datasets, and each dataset has its own target, and for every dataset the Experiment Manager finds the best combination of LSTM hyperparameters? or will I have to code the objective function and loop over it 200 times?
Jorge Calvo
Jorge Calvo 2021-5-27
Each time you run an experiment, the Experiment Manager will find the best combination of hyperparameters for a given setup. To specify what you mean by best, you can select from some standard objective metrics (including validation accuracy, which I think is what the original question was using) or you can define your own.
If you want to do find the best combo of hyperparameters for each of 200 data sets, then you would:
  1. Setup the experiment for the first data set.
  2. Run the experiment.
  3. Modify the setup function to load the next data set.
  4. Run the experiment again.
  5. Repeat steps 3 and 4.
This amounts to running 200 different experiments. On the bright side, unless your objective function depends on the data set, you would not need to recode it.

请先登录，再进行评论。

SIVA SRI
SIVA SRI 2024-9-3
To customize the architecture to classify flowers, you first replace the fully connected layer. Fully connected layers need one neuron for each output class.
To add a fully connected layer, locate the Convolution and Fully Connected section of the Layer Library. Click and drag to add a layer to the canvas.

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

产品


版本

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by