Hi,
You cannot directly optimize for the parameters you mentioned using Bayesian optimization.
A possible work around would be defining a custom optimizing function that the given parameters as input and solving them sequentially.
For example
function rmse = optimizerLoss(x,y,cv,numHid,optimizer,lr)
% Train net.
net = feedforwardnet(numHid, optimizer);
net.trainParam.lr = lr;
net = train(net, x(:,cv.training), y(:,cv.training));
% Evaluate on validation set and compute rmse
ypred = net(x(:, cv.test));
rmse = sqrt(mean((ypred - y(cv.test)).^2));
end
Use the optimizers the optimizers you mentioned sequentially. And finally DropOut is not a activation.