Simple Feedforward NNet questions
7 次查看(过去 30 天)
显示 更早的评论
Good day! I am new to neural networks and in Matlab nnet toolbox.
Basically, I want to re-implement a backpropagation feedforward neural network described from a journal/research paper. The info/parameters that I currently have are the ff: 1) it consists of 2 layers: 1 hidden, 1 output 2) 6 input neurons, 6 output neurons, 5 neurons in the hidden layer 3) the values in the input are all floating point numbers, and just binary numbers in the output 4) the activation functions are tansigmoid for the first layer, then logsigmoid for the second 5) I need to get a least mean square (LMS) error that is below 0.05. 6) no info provided about the learning rate or the momentum 7) I can generate training samples as many as needed (5000 was used)
and these are my trial codes so far:
inputs = rand([6, 6000]); outputs = randi([0,1],6, 6000); net = feedforwardnet(5); net = configure(net,inputs, outputs); net.layers{2}.transferFcn = 'logsig'; net.trainParam.epochs = 6000; net.trainParam.goal = 0.01; net = train(net,inputs,outputs);
I'm not using yet my data since I'm still thinking how to normalize my inputs. My problem is that when I try to use the network, the output is not a binary array but some floating point values. What should I do about it?
And any further thoughts/advices? I would really appreciate them. Thank you in advance.
0 个评论
采纳的回答
Greg Heath
2012-9-26
Binary numbers in the output generally indicate a classifier. In which case I advise using patternnet, unit vector column targets (help ind2vec), trainscg training function, tansig hidden and softmax output activation functions (posterior probability estimates sum to unity). When the trained net is used, the index of the assigned class is obtained from vec2ind.
If it is not a classifier, use logsig output functions. To get binary outputs you can replace logsig with hardlim AFTER training OR convert the output to unipolar {0,1} binary using round.
For a I-H-O = 6-5-6 node topology, the number of unknown weights to estimate are Nw = (I+1)*H+(H+1)*O = 35+36 = 71.
If the default data division is used, the number of training cases is Ntrn = 0.7*N and the number of training equations is Neq = Ntrn*O = 4.2*N. The rule of thumb Neq/Nw >> 1 yields N >> 71/4.2 = 17. The guideline N > 30*17 = 510 should be sufficient. N ~ 5000 may be over the top.
MSEgoal = 0.1*(Neq-Nw)*mean(var(t,0,2))/Neq
Should be sufficient as should defaults for the remaining parameters.
Hope this helps.
Thank you for formally accepting my answer.
Greg
更多回答(1 个)
Greg Heath
2012-10-2
My codes so far:
inputs = xlsread('data.xls',1);
inputs = inputs'; % 6 x 1000
outputs = xlsread('data.xls',2);
1. RENAME 'OUTPUTS' AS 'TARGETS'
outputs = outputs'; % 6 x 1000
outputs = (outputs)*2 - 1;
2, WHY DID YOU DO THIS? .
a. TO USE THE WEIGHTS OUTSIDE OF MATLAB?
OR
B. TO CAPITALIZE ON THE TANSIG HIDDEN NODES?
net = feedforwardnet(15); % OR patternnet
net = configure(net,inputs, outputs);
net.layers{2}.transferFcn = 'logsig';
3. LOGSIG IS NOT COMPATIBLE WITH [-1, 1]
...COULD USE TANSIG. THEN
y = round( (1+net(inputs)) / 2)
4. NO ...BETTER YET:
a. KEEP ORIGINAL {0,1} AND LOGSIG OUT
b. DISABLE INTERNAL OUTPUT DEFAULT NORMALIZATION {-1,1} VIA MAPMINMAX
net.trainParam.lr =0.00;
net.biasConnect(1) = 0;
net.biasConnect(2) = 0;
5. DELETE ABOVE 3 STATEMENTS
net.trainParam.goal = 0.001;
TRY = 0.01*mean(var(output'))
%[net, tr] = train(net,inputs,outputs);
[ net tr Ytrn Etrn ] = train(net,inputs,outputs)
y = round(sim(net,inputs))
6. YOU DIDN'T UNNORMALIZE THE PRETRAINING NORMALIZATION
WHAT ARE MIN AND MAX OF net(inputs) ?
% And majority of the Y values differ by 2 or more values from the actual % targets, e.g. y(:,35) = [0 0 1 1 1 1] while the actual output should be % [1 1 0 0 1 1] .
7. SEE 6
% (1) How can I further (much further) improve this? (2) Is it always % overfitting (if my understanding of overfitting is correct)?
UDERSTANDING IS INCORRECT. SEE THE COMP.AI.NEURAL-NETS FAQ.
(3) Am I % still doing something wrong with the training? Any rules of thumb, or % advices?
7. TO FIND SOME OF MY SAMPLE CODE, SEARCH NEWSGROUP AND ANSWERS USING
NEQ NW NTRIALS
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!