Neural Networks - Break down of training, testing and validation groups

Question

jlt199 2016-10-13

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/307124-neural-networks-break-down-of-training-testing-and-validation-groups

回答： Benhur Aysin 2017-9-27

Good morning,

I am trying to run a NN, for a regression type problem, with a different number of neurons each time. Then I have a measure of how each NN has performed. However, I would like to break this down into how each of the three groups (training, testing and validation) performed, but I can't get the index values of each group. Here is an excerpt from my code:

    trainFcn = 'trainbr';
    net = feedforwardnet;
    % Setup Division of Data for Training, Validation, Testing
    net.divideParam.valRatio = 15/100;
    net.divideParam.testRatio = 35/100;
    net.divideParam.trainRatio = 50/100;
    net.trainFcn = trainFcn;
    for neurons = 5:5:100
        net.layers{1}.size = neurons;
        net = configure(net,PWL',target');
        [net,tr] = train(net,PWL',target');
    end

I understood that I should be able to find the index' of the three groups using:

tr.trainInd;
tr.testInd;
tr.valInd;

However, my tr.valInd is ALWAYS empty and the ratios of tr.trainInd and tr.testInd are never as set. Can someone please tell me what I am doing wrong? Even when I leave the division of the three groups to default I don't get anything in tr.valInd. I've looked through the manual and in their example they have index values in each of the three groups, but they don't show the whole example so I can't replicate.

Please help me,

Thanks

ps. Another idea is to use divideind instead of dividerand, but I can't work out how and when to enter the index'. Help with this would also be appreciated.

2 个评论
显示无隐藏无

jlt199 2016-10-13

在 MATLAB Online 中打开

This is what I get output for tr. You can see that the division of training, test and validation data isn't as I wanted. Any ideas?

>> tr
tr = 
struct with fields:
        trainFcn: 'trainbr'
      trainParam: [1×1 struct]
      performFcn: 'mse'
    performParam: [1×1 struct]
        derivFcn: 'defaultderiv'
       divideFcn: 'divideind'
      divideMode: 'sample'
     divideParam: [1×1 struct]
        trainInd: [1×7846 double]
          valInd: []
         testInd: [1×1385 double]
            stop: 'Maximum epoch reached.'
      num_epochs: 1000
       trainMask: {[1×9231 double]}
         valMask: {[1×9231 double]}
        testMask: {[1×9231 double]}
      best_epoch: 1000
            goal: 0
          states: {1×10 cell}
           epoch: [1×1001 double]
            time: [1×1001 double]
            perf: [1×1001 double]
           vperf: [1×1001 double]
           tperf: [1×1001 double]
              mu: [1×1001 double]
        gradient: [1×1001 double]
            gamk: [1×1001 double]
             ssX: [1×1001 double]
        val_fail: [1×1001 double]
       best_perf: 39.3436
      best_vperf: NaN
      best_tperf: 1.2468e+03

Greg Heath 2016-10-14

编辑：Greg Heath 2016-10-14

I have published more than one examples of this in the NEWSGROUP. I will post at least one URL.

http://www.mathworks.com/matlabcentral/newsreader/search_results.html?dur=all&page=1&search_string=best_perf&sort=date_up

Hope this helps.

Greg

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2016-10-14

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/307124-neural-networks-break-down-of-training-testing-and-validation-groups#answer_238920

在 MATLAB Online 中打开

The bottom line is that TRAINBR has at least one bug w.r.t. datadivision.

 trainbr bug
 http://www.mathworks.com/matlabcentral/newsreader/search_results.html?dur=all&page=1&search_string=trainbr+bug&sort=date_up

You probably will get more consistent results if you use TRAINLM with MSEREG

Hope this helps.

Thank you for formally accepting my answer

Greg

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

Benhur Aysin 2017-9-27

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/307124-neural-networks-break-down-of-training-testing-and-validation-groups#answer_283359

To enable validation performance when using trainbr as the training function you must set max_fail to a positive number. 6 is usually the number for many training functions. max_fail=0 disables validation performance.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Neural Networks - Break down of training, testing and validation groups

2 个评论
显示无隐藏无

回答（2 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

Neural Networks - Break down of training, testing and validation groups

2 个评论 显示 无隐藏 无

回答（2 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

2 个评论
显示无隐藏无

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论