How to check the robustness of the Neural network model?
3 次查看(过去 30 天)
显示 更早的评论
Hi everyone,
I need your help for my project.
- I used Neural Pattern Recognition app, and have already built an NN model for classification with 4 labels. The NN model worked very well. Accuracy classification reaches more than 90%.
- However, when I want to check this model with new data (new data = the original data through an AWGN channel having a 10 dB signal-to-noise ratio (SNR). The classification result is always less than 30% accuracy.
- How to create an augmented dataset and train it?
- How to check the robustness of the Neural network model?
Please help me!!!
Code to create a neural network:
% Solve a Pattern Recognition Problem with a Neural Network
% Script generated by Neural Pattern Recognition app
%% Load data
% X - input data.
% Y - target data.
load('mydata.mat')
x = X;
t = Y;
%% Choose a Training Function
trainFcn = 'trainscg'; % Scaled conjugate gradient backpropagation.
%% Create a Pattern Recognition Network
hiddenLayerSize = 50;
net = patternnet(hiddenLayerSize, trainFcn);
% Choose Input and Output Pre/Post-Processing Functions
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};
% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand'; % Divide data randomly
net.divideMode = 'sample'; % Divide up every sample
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Choose a Performance Function
net.performFcn = 'crossentropy'; % Cross-Entropy
% Choose Plot Functions
net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
'plotconfusion', 'plotroc'};
%% Train the Network
[net,tr] = train(net,x,t);
%% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);
%% Recalculate Training, Validation and Test Performance
trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y)
valPerformance = perform(net,valTargets,y)
testPerformance = perform(net,testTargets,y)
%% View the Network and plots confusion matrix
view(net)
figure, plotconfusion(t,y)
save ('mynet');
Code to check neural network with data added white noise:
clc;
clear all;
%load
load('mynet');
%% add Gaussian white noise SNR=10dB
load('mydata.mat')
X10dB = awgn(X,10,'measured');
x = X10dB; % train data
t = Y; % target
%% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);
% Recalculate Training, Validation and Test Performance
trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y)
valPerformance = perform(net,valTargets,y)
testPerformance = perform(net,testTargets,y)
%% Plots
figure, plotconfusion(t,y)
0 个评论
采纳的回答
Greg Heath
2019-7-14
编辑:Greg Heath
2019-7-14
If you are going to test with white noise, include white noise in your design (i.e., training + validation)
Then, given a fixed input level of white noise for design (i.e., design + noise1) you can obtain individual performance measures of training, validation and test as a function of added noise level.
Hope this helps,
Thank you for formally accepting my answer
Greg
更多回答(0 个)
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!