a problem with neural network training

2013 12 4
1 个回答
回答已采纳
更新时间：2013 12 6
3 次查看（30 天）

0 个投票

Hi everybody I have read in some papers that in order to avoid your neural network getting stuck in local minima during resampling methods, a network is trained on the entire data set to obtain a model ) with weights W0, then, these weights are used as the starting point for vtraining the other samples. I have applied this method on a dataset but, while trying to train the second sample the training is not being completed. I have changed some of the neural network parameters like min_grad periodically however, it did not solve this problem. Is it due to the fact that our initial guess is so close to the minimum? Is the problem related to my code? or there is other problem in this field. The code is as follows: [x,t]=house_dataset; Inputs=cell(1,2); Targets=cell(1,2); for i=1:2 Ind=randi(size(x,2),size(x,2),1); Inputs{i} = x(:, Ind); Targets{i}=t(Ind); end NN=cell(1,2); net=feedforwardnet; rng(0); NN{1}=train(net,Inputs{1},Targets{1}); IW = NN{1}.IW; LW = NN{1}.LW; B = net.b; NN{2}.IW = IW; NN{2}.LW = LW; NN{2}.b = B; net.initFcn=''; NN{2}=train(net,Inputs{2},Targets{2});
Your help is greatly appreciated.

请先登录，再回答此问题。

 采纳的回答

Greg Heath
Greg Heath 2013-12-5

0 个投票

There are MANY thing wrong with your code.
1. Your premise of using a full data design to initialize divided data designs doesn't make sense.
2. Your data selection using randi randomly selects data with replacement. Therefore ,e.g., you may only have 325 unique examples out of 506. The rest are duplicates and 181 are not used at all!
3. The default data division applied to the non-unique data selection not only results in training, validation and test subsets with duplicated data, some data points belong to more than one of the 3 subsets.
4. You train and use a net, NN{1} without checking to see if the design is any good.
5. I see no reason to put networks inside cells.
Thank you for formally accepting my answer
Greg
P.S. Search using
greg fitnet Ntrials % regression/curvefitting
greg patternnet Ntrials % classification/pattern-recognition
for some of my sample code.

2 个评论
显示 无 隐藏 无

Mohamad
Mohamad 2013-12-5
Dear Greg thank you very much.
Considering your answers some of the reasons for using this code are as follows:
1. My premise is based on the following references:
Moody, J. 1994. Prediction Risk and Architecture Selection for Neural Networks,….
Utans, J. and Moody, J. (1991), Selecting neural network architectures via the prediction risk: Application to corporate bond rating prediction, in ‘Proceedings of the First International Conference on Artificial Intelligence Applications on Wall Street’, IEEE Computer Society Press, Los Alamitos, CA.
Moody, J. E. and Utans, J. (1992), Principled architecture selection for neural networks: Application to corporate bond rating prediction, in J. E. Moody, S. J. Hanson and R. P. Lippmann, eds, ‘Advances in Neural Information Processing Systems 4’, Morgan Kaufmann Publishers, San Mateo, CA, pp. 683–690.
2. My data selection is the same procedure which is used by bootstrap method, sampling with replacement, or at least I assume that they are the same.
3. Yes, I agree with you.
4. When I store the NNs in different cell, I can refer to each one of them and consider their performance after training for all of them is complete.
best
Greg Heath
Greg Heath 2013-12-6
1. Thanks for the references. However, I don't have access to them. Moreover, since the dates are ancient, I'll stand by my skepticism.
2. Sampling with replacement does not mean that you should not try to use all of the data in each replication. The last subset should contain all of the data not chosen by the other subset(s). However, you can get away with it if you have a huge data set AND a huge number of replications. Otherwise, you should try to use all of the data in each replication.
4. I don't object to storing nets in a cell. However if you use the command whos, you will see that your second net is not recognized as a net.

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by