Mapminmax process function causes that NN incorrectly simulates outputs

Question

Céldor 2016-4-6

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/277497-mapminmax-process-function-causes-that-nn-incorrectly-simulates-outputs

评论： Brendan Hamm 2016-4-12

Hello,

I have a problem with some outputs from a trained custom neural network. I am using MATLAB 2014 with NN ToolBox ver 8.2.

I have created a simple feedforward NN for classification. I have used some Inputs and Targets, trained the NN, and tried to simulate Outputs given Inputs from the same range. The NN I created gives me incorrect outputs. First it returns outputs only with 0,1 range while the targets are in rage of -6 ... 3 and when I add a process function 'mapminimax' to input and output, the results are wrong: 1. when targets are 0 ... 1, there is an offset of 0.5 such that the outputs are 0.5 and 1 2. when targets are e.g. -6 ... 3, the outputs are around -3 ... 3

I am trying to understand what I am doing wrong.

PS. I have already asked this question in SO and also I provided some more details code: http://stackoverflow.com/questions/36449224/mapminmax-process-function-causes-that-nn-incorrectly-simulates-outputs

The code to test NN

    clear all; close all; clc;
    net = NNPatRec;
    %net.inputs{1}.processFcns = {'mapminmax'};
    %net.outputs{2}.processFcns = {'mapminmax'};
    Inputs = -10:10;
    %Targets = [-6*ones(1,11) 3*ones(1,10)];
    Targets = [zeros(1,11) ones(1,10)];
    [net,tr] = train(net,Inputs,Targets);
    net(-10:10)

The code to create NN:

    function net = NNPatternRecognition
        net = nntest;
    end
    function net = nntest
        net = network;
        net.numInputs = 1;
        net.numLayers = 2;
        net.biasConnect = [1 1]';
        net.inputConnect = [1; 0];
        net.layerConnect = [0 0; 1 0];
        net.outputConnect = [0 1];
        % Inputs
        %net.inputs{1}.processFcns = {'mapminmax'};
        net.inputWeights{1}.learnFcn = 'learngdm';
        % layers 1 (at input)
        net.layers{1}.initFcn = 'initnw';
        net.layers{1}.netInputFcn = 'netsum';
        net.layers{1}.transferFcn = 'tansig';
        net.layers{1}.size = 3;
        % layers 2 (hidden)
        net.layers{2}.initFcn = 'initnw';
        net.layers{2}.netInputFcn = 'netsum';
        net.layers{2}.transferFcn = 'purelin';
        net.layers{2}.size = 1;
        % Network functions
        net.adaptFcn = 'adaptwb';
        net.derivFcn = 'defaultderiv';
        net.divideFcn = 'dividerand'; %'divideblock';
        net.initFcn = 'initlay';
        net.performFcn = 'crossentropy';
        net.trainFcn = 'trainscg';
        % Outputs
        %net.outputs{2}.processFcns = {'mapminmax'};
        %net.outputs{2}.exampleOutput = [0 1];
        net.trainParam.showWindow = false;
        net.trainParam.showCommandLine = true;    
    end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Greg Heath 2016-4-8

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/277497-mapminmax-process-function-causes-that-nn-incorrectly-simulates-outputs#answer_217034

在 MATLAB Online 中打开

 close all, clear all, clc, plt=0, tic
 x       = -10:10;  N = length(x)
 trueind = 1 + [zeros(1,11) ones(1,10)];
 t       = full(ind2vec(trueind))
 plt = plt+1, figure(plt), hold on
 plot( x( 1:11), trueind(  1:11) ,'o' )
 plot( x(12:21), trueind(12:21),'ro' )
 axis([ -11 11 0 3 ])
 title('CLASS INDICES')
 rng('default')
 net            = patternnet;
 [ net tr y e ] = train( net, x, t );
 outind         = vec2ind(y)
 plot( x( 1:11), outind(  1:11) ,'x' ,'LineWidth',2)
 plot( x(12:21), outind(12:21),'rx' ,'LineWidth',2)
 err    = outind~=trueind;
 Nerr   = sum(err)           % 1
 PctErr = 100*Nerr/N       % 4.7619
 Hope this helps.
 For details, remove the semicolon to get
 net = net

ALSO, for a trn/val/tst breakdown use

tr = tr

Hope this helps.

Thank you for formally accepting my answer

Greg

3 个评论
显示 1更早的评论隐藏 1更早的评论

Greg Heath 2016-4-12

I don't know where you got that idea. Did you run my program?

Brendan Hamm 2016-4-12

There should be exactly 2 rows in the output. The probability of Class_1 and the probability of Class_2. You just refer to these as Class_0 and Class_1 respectively.

请先登录，再进行评论。

Answer 2

Brendan Hamm 2016-4-6

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/277497-mapminmax-process-function-causes-that-nn-incorrectly-simulates-outputs#answer_216764

在 MATLAB Online 中打开

You would likely have better luck if you just started with the patternnet which is meant for NN classification.

What I see that is wrong with your current implementation is you have a linear transferFcn for your second layer. For classification purposes this should really be a softmax function. That is change

net.layers{2}.transferFcn = 'purelin';

to

net.layers{2}.transferFcn = 'softmax';

There are also 2 functions used for the processFcns for the the input and output:

net.outputs{2}.processFcns == {'removeconstantrows', mapminmax};
net.inputs{1}.processFcns == {'removeconstantrows', mapminmax};

4 个评论
显示 2更早的评论隐藏 2更早的评论

Brendan Hamm 2016-4-7

在 MATLAB Online 中打开

mapminmax will map the output values to a minimum and maximum value. By default this is set to the range [-1 1]. So, I suppose I missed this part above, we would need to change the range to be [0,1]. This is controlled in the processParams:

p.outputs{2}.processParams{1}.max_range = 0; % params for remove constant rows
p.outputs{2}.processParams{2}.ymin = 0; % Params for mapminmax
p.outputs{2}.processParams{2}.ymax = 1; % Params for mapminmax

Furthermore with a classification problem the output should really be a probability of belonging to each class. For this reason for a binary classification you would expect 2 outputs for every sample, the probability of class1 and the probability of class2.

So, You really want your targets to be a matrix of ones and zeros where 1 indicates belonging to that class (i.e. you know it belongs to class1 so the probability is 1). You can get this result by calling:

dummyvar(Targets+1)';

I use the +1 as dummyvar expects all inputs to be positive integers (or alternatively categorical or strings) So all in all this could have been accomplished with the following:

Inputs = -10:10;
Targets = [ones(1,11) 2*ones(1,10)]; % I'll Increment by 1 here (Class name is irrelevant)
p = patternnet(10); % For classification problems
Targets = dummyvar(Targets)'; % Make this into an indicator matrix
p = train(p,Inputs,Targets); % train the NN
Tpred = p(Inputs) % Verify the results with our training set.
Tpred =
Columns 1 through 3
  1.0000    1.0000    1.0000
  0.0000    0.0000    0.0000
Columns 4 through 6
  1.0000    1.0000    1.0000
  0.0000    0.0000    0.0000
Columns 7 through 9
  1.0000    1.0000    1.0000
  0.0000    0.0000    0.0000
Columns 10 through 12
  1.0000    1.0000    0.0000
  0.0000    0.0000    1.0000
Columns 13 through 15
  0.0000    0.0000    0.0000
  1.0000    1.0000    1.0000
Columns 16 through 18
  0.0000    0.0000    0.0000
  1.0000    1.0000    1.0000
Columns 19 through 21
  0.0000    0.0000    0.0000
  1.0000    1.0000    1.0000

Brendan Hamm 2016-4-7

As a shameless plug, if you ever want your Machine Learning Algorithms to feel a little bit less "black-box", consider attending the Machine Learning with MATLAB course.

Céldor 2016-4-8

编辑：Céldor 2016-4-8

Many thanks for this Brendan. Everything's working :)

Regarding mapminmax, I also found what I needed to set yesterday. I had to go through the process of how the command patternnet creates a a netowrk and realised it uses feedforwardnet. But somewhere in the code, there's a line where the ymin is set to 0. I have set ymin by myself and everything started to work :)

And thanks for the comment on preparation of Targets for classification. This is another thing I had to go through by myself and realised that NN works much better if there are multiple outputs, each representing individual class.

Thanks :)

PS. I have just learned a new command: dummyvar

请先登录，再进行评论。