How to choose the appropriate trained NN?

1 次查看（过去 30 天）
TS Sharma
TS Sharma 2014-12-21
评论： TS Sharma 2014-12-22
Hi!
I have divided my data into trn/tst/val sets. The NN gives different classification accuracy at every training session.Should I choose my model simply based on the highest test set accuracy or should I average the test accuracy over several runs?
Thanks in advance.

请先登录，再回答此问题。

采纳的回答

Greg Heath
Greg Heath 2014-12-21
Doesn't make sense to average error rates in order to choose the best design.
Obtain multiple designs(e.g., ~100: ~10 for each of 10 choices for number of hidden nodes)
Rank them via the degree-of-freedom adjusted training set error and validation set error
Estimate generalization non-design error via the test set error
I tend to choose the smallest successful net by just looking at the three tabulations and plots of error vs # of hidden nodes.
Hope this helps
Thank you for formally accepting my answer
Greg
  3 个评论
显示 1更早的评论
Greg Heath
Greg Heath 2014-12-22
Different people have different definitions.
I associate model selection with topology, number of layers, choice of transfer functions, ...
I associate parameter selection with a selection of values consistent with topology and optimization of a performance function.
If you type the command (without the ending semicolon)
net = patternnet
you will see the list of defaults chosen for both.
To begin with, run the help and doc examples. Model and parameter selection are automatically chosen by default. If performance is unsatisfactory just run the example Ntrials = 10 or more times to mitigate the random choice of initial weights and trn/val/tst data division. If that fails, increase the number of hidden nodes obtaining Ntrials = 10 or more designs for each candidate value of number of hidden nodes.
For a structured approach to this technique, search including the search words
greg Ntrials.
Hope this helps.
Thank you for formally accepting my answer
Greg

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Deep Learning Toolbox 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by