RNG neural Network and outputs

6 次查看（过去 30 天）
Andreas
Andreas 2016-2-24
评论： Andreas 2016-3-5
Hello everyone.!
I am new in nn so that might be a silly question, but as I change the rng of my nn, the quality of the solution changes too.
For example, for a specific rng , the best setup is using softmax in the second layer, with 11 neurons in the first one. However with a different rng, the best setup is logsig in the second layer with 11 neurons in the first one. What is going on with that?Is there an optimal rng? Also, although I have formulated my output in a 1-c form, the output I get is not binary. Why? I use patternnet, with 10 input categories with 180 responses and 5 output classes.
Thank you all.!

请先登录，再回答此问题。

采纳的回答

Greg Heath
Greg Heath 2016-2-27
GEH1 = 'Size of input and target matrices and Hub?'
GEH2 = ' Are target columns {0,1} unit vectors?'
GEH3 = 'I find it better to only look at ~ 100 designs at a time : Ntrials ~ numel(Hmin:dH:Hmax) ~ 10. I don"t recall ever having to make more than ~ 200 designs'
GEH4 = ' Too much space is wasted on statements that assign defaults. Accept as many as you can and delete the corresponding statements. goal and min_grad are the only ones I specify'
GEH5 = ' Use dH > 1 and do not use h as a matrix index'
GEH6 = 'defaults TRAINSCG and CROSSENTROPY are usually preferred for classification'
GEH7 = ' Error rates are what what you are trying to minimize. Check out some of my patternnet posts that yield error rates.'
ope tis helps.
Thank you for formally accepting my answer
Greg
  3 个评论
显示 1更早的评论
Greg Heath
Greg Heath 2016-2-28
% I=10, N=173, O=5, Ntrn=121, Ntrneq=605, Hub=37.5
Ntrials = 10, h = 0:4:36 or 1:4:37 yields ~ 100 values.
% Targets were constructed through dummyvar command,
WHAT?
indices = [ 5 3 1 4 2 ],
target1 = full(ind2vec(indices)) % Correct
target2 = dummyvar(indices) % Error
% Maybe I could use this: % Ndof = Ntrneq-Nw % MSEgoal = 0.01*(Ndof/Ntreq)*MSE00 % (MSE00=mean(var(targets',1)) ) , % as a goal?
No. Only use training data
MSEtrn00 = mean(var(targets(:,trnind))',1) % biased
MSEtrn00a = mean(var(targets(:,trnind))',0) % unbiased
Ntrndof = Ntrneq-Nw
MSEgoal = 0.01*max(0,Ntrndof/Ntrneq)*MSEtrn00a
% Finally, using: % % input=minmax(std(input)) % could give me a better quality of input for my ordinal data?
No. That makes no sense.
(Do you mean integer data?)
% By error rate you mean the sum(target~=predicted)?
No. That is Nerr, the total number of errors.
PctErr = 100*Nerr/N
However, you also have to calculate the errors for trn/val/tst using the indices in tr
% I've read that adding noisy duplicates of small classes with % high error rates can address the problem, but I cannot get %the implementation of it.
Something like
noisyduplicate = mean + a*std*randn(size(original)) % scalar
for each input variable (different means and stds).
Good to have Ni (i=1:c) equal
See my BIOID posts
% Also I chose trainlm as it had less overall error (~0.21) than trainscg (~0.31)
Again, val/tst error rates should be the criteria
Hope this helps.
Greg
Andreas
Andreas 2016-3-5
Dear Dr. Heath,
I did change my error measurement to
sum(Output2(tr.trainInd)'~=vec2ind(y(:,tr.trainInd)))+sum(Output2(tr.valInd)'~=vec2ind(y(:,tr.valInd)))
in order to get the net with the minimum overall errors, and it seems to work good. About the dummyvar, I used the ' sign in order to get the same result with ind2vec.
I don't have integer data, I have categorical ones, so the use of mapstd or minimax is pointless isn't it?
Could the use of trainlm be a mistake?No matter what I've tried trainlm seems to give the best overall results.
Finally, the best I can get is 15% error, although I have followed every suggestion you and others posted.
Could that be the maximum result I can get?
I post you some of my code.
It is always an honor.
Best regards, Andreas Kampianakis
inputs = Input';
targets = dummyvar(Output2)';
Hmin=1;
dH = 4;
Hmax = 37;
for h = Hmin:dH:Hmax
% Create a Pattern Recognition Network
hiddenLayerSize = h;
% [ I N ] = size(inputs) ;
% [ O N ] = size(targets);
% Ntrn = N -2*round(0.15*N);%
% Ntrneq = Ntrn*O ;%
% Nw = (I+1)*h+(h+1)*O;
% Ndof = Ntrneq-Nw
net = patternnet(hiddenLayerSize);
%
% net.inputs{1}.processFcns = {'removeconstantrows','fixunknowns'};
% net.outputs{2}.processFcns = {'removeconstantrows'};
% net.outputs{2}.processFcns={};
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'softmax';
% MSEgoal = 0.01*mean(var(target',1));
% MinGrad = MSEgoal/100;
% net.trainParam.goal = MSEgoal;
% net.trainParam.min_grad = MinGrad;
% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand';
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
net.trainFcn = 'trainlm';
net.trainParam.max_fail= 6;
net.trainParam.showWindow=false;
net.performFcn = 'mse';
% Train the Network
for i = 1: 15
net = configure( net, inputs, targets);
[net,tr, y, e] = train(net,inputs,targets); %br gamaei
Kapa=sum(Output2(tr.trainInd)'~=vec2ind(y(:,tr.trainInd)))+sum(Output2(tr.valInd)'~=vec2ind(y(:,tr.valInd)));
if Kapa<BKapa
Hiddennodes = h;
BKapa=Kapa;
Btr=tr;
By=y;
Bnet=net;
end
end
end

请先登录，再进行评论。

更多回答（1 个）

Greg Heath
Greg Heath 2016-2-25
I typically
1. Initialize rng once and only once before training (henever you use the
rng it AUTOMATICALLY moves to another state).
2. Use a double for loop design over ~ 10 candidates for H, number of
hidden nodes and ...
3. For each value of H, design ~ 10 candidate nets with different random
data divisions AND random intitial weights.
4. for h = Hmin:dH:Hmax
net = .
for i = 1:Ntrials ...
I have posted zillions of examples on both the NEWSGROUP and ANSWERS. Search using
patternnet Ntrials
or
patternnet Hmin:dH:Hmax
Hope this helps.
Thank you for formally accepting my answer
Greg
  1 个评论
Andreas
Andreas 2016-2-25
编辑：Andreas 2016-2-25
Dear Dr. Heath,
thank you for your answer, it is an honor.
I have read many of your answers at ANSWER and NEWSGROUP, and this is my final piece of code. Although I have figured out a seemingly optimal nn, it does not seem to work at all. With R2 having a value of 0.9280 at max, I think its accuracy is not adequate. My inputs are nominal {1-5}, with 10 categories, from 182 responses and my output is categorical data with 5 classes, formulated through 1-c (using dummyvar). What other options do I have in order to gain better results?
Thank you Dr. Heath.
rng(4500)
inputs = Input';
targets = target';
Hmin=1;
dH = 1;
Hmax = 40;
Ntrials = 50;
for h = Hmin:dH:Hmax
% Create a Pattern Recognition Network
hiddenLayerSize = h;
net = patternnet(hiddenLayerSize);
net.inputs{1}.processFcns = {'removeconstantrows','fixunknowns','mapstd'};
net.outputs{2}.processFcns = {'removeconstantrows'};
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'softmax';
% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand';
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
net.trainFcn = 'trainlm';
net.trainParam.max_fail= 6;
net.performFcn = 'mse';
net.trainParam.epochs=1000;
net.trainParam.min_grad=0;
for i = 1: Ntrials
net = configure( net, inputs, targets);
[net,tr, y, e] = train(net,inputs,targets);
stopcrit{i,h} = tr.stop;
bestepoch(i,h) = tr.best_epoch;
NMSE = mse(targets-y)/MSE00;% Normalization
R2(i,h) = 1-NMSE; % Rsquare: See Wikipedia
end
end
stopcrit
bestepoch
R2
% Test the Network
outputs = net(inputs);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs);

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Pattern Recognition 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by