Optimization of hidden layer dimension in neural network (number of neurons)

Question

David Franco 2017-12-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/372968-optimization-of-hidden-layer-dimension-in-neural-network-number-of-neurons

编辑： PIYUSHA GARG 2020-6-12

I am using GA optimization to train my neural network (to try to find the best weights set):

% Neural Network trained by Genetic Algorithm (GA)
% Reference: http://www.mathworks.com/matlabcentral/answers/100323
% Thx Greg!
rng('default')
[x, t] = simplefit_dataset;
[I, ~] = size(x);
[O, N] = size(t);
% Reference MSE: Average Target Variance
var_t = mean(var(t,1,2));
% Hidden Node Choice
H = 4;
Nw = (I + 1)*H + (H + 1)*O;
% Create Regression/Curve-Fitting Neural Network:
net = feedforwardnet(H);
net.divideFcn = 'dividetrain';
% Configure the Net for the Simplefit Dataset
net = configure(net, x, t);
% Initial Weights and Errors
wb = getwb(net)';
% Create handle to the error function,
fun = @(wb) nmse(wb, net, x, t); % NMSE
% Set the Genetic Algorithm tolerance for minimum change in fitness function
% before terminating algorithm to 1e-4 and display each iteration's results.
opts = optimoptions('ga', 'FunctionTolerance', 1e-4, 'Display', 'iter');
tic
[wbopt, fval] = ga(fun, Nw, opts);
totaltime = toc;
% Assign the weights to net
net = setwb(net, wbopt');
% Simulate the output
y = sim(net,x);

Where the function nmse is:

function nmse_calc = nmse(wb, net, input, target)
      % 'wb' contains the weights and biases vector
      % in row vector form as passed to it by the
      % genetic algorithm. This must be transposed
      % when being set as the weights and biases
      % vector for the network.
      % Reference MSE
      var_t = mean(var(target,1,2));
      % To set the weights and biases vector to the
      % one given as input
      net = setwb(net, wb');
      % To evaluate the ouputs based on the given
      % weights and biases vector
      y = net(input);
      % Calculating the Normalised Mean Squared Error (NMSE)
      nmse_calc = mean((target(:) - y(:)).^2) / var_t;
end

My question is:

Is there any way to optimize the number of hidden nodes H in the same way? Maybe with multiobjective GA optimization... optimizing the weights set and the number of hidden nodes at once.

Thank you very much!

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

PIYUSHA GARG 2020-6-12

编辑：PIYUSHA GARG 2020-6-12

please tell how to write Nw for multiple hidden layers.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2017-12-20

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/372968-optimization-of-hidden-layer-dimension-in-neural-network-number-of-neurons#answer_297133

编辑：Greg Heath 2017-12-20

在 MATLAB Online 中打开

If you search the NEWSGROUP and ANSWERS using

greg genetic

you should conclude that I don't recommend GA for NN design.

Nevertheless, if you insist, I recommend a double loop design:

 j =0
 for h = Hmin:dH:Hmax
   j=j+1
   standard stuff
   for i = 1:Ntrials
   GA
   end
 end

Hope this helps,

Thank you for formally accepting my answer

Greg

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

David Franco 2017-12-28

编辑：David Franco 2017-12-28

But in this way, there is no optimization of the number of Hidden nodes through a global optimization technique, just an exhaustive check of all Hidden nodes possibilities.

The number o Hidden nodes must be optimized too (as an input for GA, for instance)...

请先登录，再进行评论。

Optimization of hidden layer dimension in neural network (number of neurons)

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

Optimization of hidden layer dimension in neural network (number of neurons)

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

采纳的回答

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

更多回答（0 个）

另请参阅

类别

标签

Community Treasure Hunt

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论