Implementation of the feedforwardnet Neural Network

34 次查看（过去 30 天）
SMEAC
SMEAC 2021-2-17
评论： SMEAC 2021-2-22
I have implemented a very simple neural network to estimate a sine function. The following is the code for generating and training the network:
% Generate Data
dataSize = 1000;
x = linspace(0, 2*pi, dataSize);
y = sin(x);
hold off
plot(x,y)
hold on
% Add noise to Data
yInput = y+randn(1,dataSize)./5;
% No need to seperate training, test and validation data, that occurs automatically in train function.
% Generate Network. A Very simple two layer model, with two nodes in input layer.
net = feedforwardnet([2]);
% Train Network
net = train(net,x,yInput);
% Show result of trained network
yNN = net(x);
figure
plot(x,yNN, '*')
hold on
plot(x,y, '.')
Now my question, how is this network actually implemented. According to literature, I should be able to recreate the network, copying the weights and biases, with the following function:
function [y] = mynet(net, x_val)
%MYNET A manual implementation of the feedforward network, to demonstrate functionality.
W1 = net.IW{1};
b1 = net.b{1};
W2 = net.LW{2};
b2 = net.b{2};
y = purelin(W2*tansig(W1*x_val + b1)+b2);
end
However, the original net(x) function, and mynet(x) produce completely different results. Although the weights and biases are exactly the same, the functions are also directly copied over, you can extract them from the network with:
>> net.layers{1}.transferfcn
ans =
'tansig'
>> net.layers{2}.transferfcn
ans =
'purelin'
Can anyone suggest where my implementation of the neural network is wrong. I am really hoping that it is a simple mistake, but I just cant see it at the moment.
Many thanks in advance
  2 个评论
Joshua Ogbebor
Joshua Ogbebor 2021-2-21
Hi
Can you run the code net.inputWeights{1,1}.weightFcn and see if it is dot product?
SMEAC
SMEAC 2021-2-21
编辑：SMEAC 2021-2-21
Hello,
yes, I can confirm it is the dot product.
>> net.inputWeights{1,1}.weightFcn
ans =
'dotprod'
And this is essentially what is already programmed. My network only has one input, so the input value x_val is a scalar. And no dot product is calculated. However, moving onto the second layer, where the different nodes with the different weights are brought together, this is evaluated with the vector vector multiplication W2*tansig... The following example shows this is calculating the same:
>> W2*tansig(W1*x_val + b1)
ans =
0.5891
>> dot(W2,tansig(W1*x_val + b1))
ans =
0.5891

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Steven Lord
Steven Lord 2021-2-22
Ran in:
How are you handling the preprocessing functions and postprocessing functions?
net = feedforwardnet([2]);
net.inputs{1}.processFcns
ans = 1x2 cell array
{'removeconstantrows'} {'mapminmax'}
net.outputs{2}.processFcns
ans = 1x2 cell array
{'removeconstantrows'} {'mapminmax'}
See the description of the inputs, outputs, etc. subobjects on this documentation page for more information.
  1 个评论
SMEAC
SMEAC 2021-2-22
Yes!! Many thanks.
I hadn't seen that the processFcns work as pre- post-processing functions.
On my network, the
net.inputs{1}.processFcns and net.outputs{2}.processFcns both give a single cell answer of the mapminmax function.
When I modify my code to include this it works perfectly.
For Info, in my implementation, I have chosen not to use directly the mapminmax function, because it requires that the entire range is given as an input, making it impossible to use the network to evaluate a single output. But as it is a linear mapping function, it is essentially just a further gain and offset, or weight and bias. :)

请先登录，再进行评论。

更多回答（1 个）

Joshua Ogbebor
Joshua Ogbebor 2021-2-21
In general the code net(x) is equivalent to sim(net, x), and this implementation depends on so many properties of the feedforward nnetwork that you may not be aware of. I copied this from the sim function page and thought t would be helpful. You could investigate and see if any one of these could be the issue.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
sim uses these properties to simulate a network net.
net.numInputs, net.numLayers
net.outputConnect, net.biasConnect
net.inputConnect, net.layerConnect
These properties determine the network’s weight and bias values and the number of delays associated with each weight:
net.IW{i,j}
net.LW{i,j}
net.b{i}
net.inputWeights{i,j}.delays
net.layerWeights{i,j}.delays
These function properties indicate how sim applies weight and bias values to inputs to get each layer’s output:
net.inputWeights{i,j}.weightFcn
net.layerWeights{i,j}.weightFcn
net.layers{i}.netInputFcn
net.layers{i}.transferFcn
  1 个评论
SMEAC
SMEAC 2021-2-21
I have been looking at most of these options, but they dont make much of an influence in the implementation here.
For example, all delays are 0:
>> net.inputWeights{1}.delays
ans =
0
>> net.layerWeights{2}.delays
ans =
0
The input function is the dot product what I discussed above and the transfer functions are the tansig and purelin which are implemented.
To use the slides from Prof. Ricardo de Castro from the TU Muenchen, the implemenation of sim(net, x) and net(x) as well as my implementation of mynet(x) should all be exactly the same based on the following description of a basic neural network.
Which in the MATLAB Code is shown by:
Unfortunately, the actual implementation in matlab is hidden in nnMex.yy() a compiled mex function where the code is not visible. So I cant see why I have made a mistake or how they come to their conclusion.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Define Shallow Neural Network Architectures 的更多信息

标签

产品


版本

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by