How to simulate default patternnet with feedforwardnet in Matlab?
显示 更早的评论
I got very different training efficiency with the following network
net = patternnet(hiddenLayerSize);
and the following one
net = feedforwardnet(hiddenLayerSize, 'trainscg');
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'softmax';
net.performFcn = 'crossentropy';
on the same data.
I was thinking networks should be the same.
What thing I forgot?
UPDATE
The code below shows, that patternnet is systemtically outperforms feedforwardnet. This proves that feedforwardnet is initilized differently somehow. The question is what is the difference?
hiddenLayerSize = 10;
% pass 1, with patternnet
net = patternnet(hiddenLayerSize);
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
[net,tr] = train(net,x,t);
y = net(x);
performance = perform(net,t,y);
fprintf('pass 1, patternnet, performance: %f\n', performance);
fprintf('num_epochs: %d, stop: %s\n', tr.num_epochs, tr.stop);
% pass 2, with feedforwardnet
net = feedforwardnet(hiddenLayerSize, 'trainscg');
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'softmax';
net.performFcn = 'crossentropy';
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
[net,tr] = train(net,x,t);
y = net(x);
performance = perform(net,t,y);
fprintf('pass 2, feedforwardnet, performance: %f\n', performance);
fprintf('num_epochs: %d, stop: %s\n', tr.num_epochs, tr.stop);
% pass 1, with patternnet
net = patternnet(hiddenLayerSize);
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
[net,tr] = train(net,x,t);
y = net(x);
performance = perform(net,t,y);
fprintf('pass 3, patternnet, performance: %f\n', performance);
fprintf('num_epochs: %d, stop: %s\n', tr.num_epochs, tr.stop);
% pass 2, with feedforwardnet
net = feedforwardnet(hiddenLayerSize, 'trainscg');
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'softmax';
net.performFcn = 'crossentropy';
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
[net,tr] = train(net,x,t);
y = net(x);
performance = perform(net,t,y);
fprintf('pass 4, feedforwardnet, performance: %f\n', performance);
fprintf('num_epochs: %d, stop: %s\n', tr.num_epochs, tr.stop);
Output follows:
pass 1, patternnet, performance: 0.116445
num_epochs: 353, stop: Validation stop.
pass 2, feedforwardnet, performance: 0.693561
num_epochs: 260, stop: Validation stop.
pass 3, patternnet, performance: 0.116445
num_epochs: 353, stop: Validation stop.
pass 4, feedforwardnet, performance: 0.693561
num_epochs: 260, stop: Validation stop.
3 个评论
Andy Zhang
2015-4-11
Hi,Dear Olga Sorry to tell you i have no idea about your question. Recently,I am using function feedforwardnet to creat a Neural Network which is used to classify the facial expression ,but disappointed ,A question 'out of memory ,TYPE HELP MEMORY for your options'hapend when i train the Neural Network and the question is not exist when i use the function patternnet or newff. Do you have some ideas about my question? Thanks.
Olga Lodnikova
2015-4-17
Greg Heath
2015-4-26
编辑：Greg Heath
2015-10-21
If
{I N ] = size(input)
[O N ] = size(target)
and the network node topology is I-H-O, then if the number of training examples is
Ntrn = (2/3)*N % default is ~ 0.7*N
the number of training equations is
Ntrneq = (2/3)*N*O
The number of unknown weights to estimate is
Nw = (I+1)*H+(H+1)*O
Therefore, the number of equations is at least equal to the number of unknowns when
Ntrn >= Ntrnlb = 1.5*Nw/O % lb => lower bound
So, typically, robust designs result when
Ntrn >~ 10*Ntrnlb ~ 15*Nw/O
Consequently, keeping this in mind can prevent the use of more data than is nececessary.
Hope this helps.
Greg
采纳的回答
Greg Heath
2015-4-25
编辑：Greg Heath
2015-4-27
When comparing numerical designs, you should initialize the RNG to the same initial state before each training.
To compare the properties of FEEDFORWARDNET and PATTERNNET, compare the screen outputs of the following commands WITHOUT THE ENDING SEMICOLON
net1 = feedforwardnet
net2 = patternnet
In particular, compare the following NINE fields
transferFcn = net.layers{2}.transferFcn
yminparam = net.outputs{2}.processParams{2}.ymin
yminsetting = net.outputs{2}.processSettings{2}.ymim
yrangesetting = net.outputs{2}.processSettings{2}.yrange
performFcn = net.performFcn
plotFcns = net.plotFcns
plotParams = net.plotParams
trainFcn = net.trainFcn
trainParams = net.trainParam
Therefore SIX additional fields besides the transferFcn, performFcn and trainFcn must be considered when trying to convert FEEDFORWARDNET to PATTERNNET and vice versa.
Hope this helps.
Thank you for formally accepting my answer
Greg
1 个评论
Greg Heath
2015-4-26
I have verified Olga's performance results using her code on the iris_dataset with and without explicit RNG initialization (ERI) USING rng(0) before training.
WITHOUT ERI
pass 1, patternnet, performance: 0.036678
num_epochs: 18, stop: Validation stop.
pass 2, feedforwardnet, performance: 0.491614
num_epochs: 15, stop: Validation stop.
pass 3, patternnet, performance: 0.036678
num_epochs: 18, stop: Validation stop.
pass 4, feedforwardnet, performance: 0.491614
num_epochs: 15, stop: Validation stop.
USING rng(0)
pass 1, patternnet, performance: 0.033005
num_epochs: 17, stop: Validation stop.
pass 2, feedforwardnet, performance: 0.490139
num_epochs: 15, stop: Validation stop.
pass 3, patternnet, performance: 0.033005
num_epochs: 17, stop: Validation stop.
pass 4, feedforwardnet, performance: 0.490139
num_epochs: 15, stop: Validation stop.
Again the patternnet performance is better
NONETHELESS, I HAVE BEEN ABLE TO MODIFY OTHER PROPERTIES TO CONVERT FEEDFORWARDNET TO PATTERNNET AND VICE VERSA TO ACHEIVE THE EXACT SAME RESULTS.
HOWEVER, I CANNOT FIND THE FILE. IT TOOK QUITE A WHILE TO FIGURE IT OUT. SO IF I CAN'T FIND IT, IT WILL BE SOMETIME BEFORE I FEEL UP TO DUPLICATING THE WORK.
SORRY,
GREG
PS: If you type the commands WITHOUT THE ENDING SEMICOLON
net = feedforwardnet
net = patternnet
and compare the properties, you can eventually figure it out.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Pattern Recognition 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
