How Can I Modify Weights in a SeriesNetwork of a CNN Model ?

25 次查看（过去 30 天）
Reza Akbari
Reza Akbari 2018-2-20
评论： Ali Al-Saegh 2021-3-10
Hi, Everyone. I have run a CNN example of Matlab R2017b to classify images of digits 0-9 and the code of this example is shown below :
if true
% Loading Data and Splitting them to the Training and Testing Data Set
digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos',...
'nndatasets','DigitDataset');
digitData = imageDatastore(digitDatasetPath,...
'IncludeSubfolders',true,'LabelSource','foldernames');
trainingNumFiles = 750;
rng(1) % For reproducibility
[trainDigitData,testDigitData] = splitEachLabel(digitData,...
trainingNumFiles,'randomize');
% Defining Layers
layers = [imageInputLayer([28 28 1]);
convolution2dLayer(5,20);
reluLayer();
maxPooling2dLayer(2,'Stride',2);
fullyConnectedLayer(10);
softmaxLayer();
classificationLayer()];
% Training the convnet
options = trainingOptions('sgdm','MaxEpochs',20,...
'InitialLearnRate',0.0001);
convnet = trainNetwork(trainDigitData,layers,options);
end
Actually, I want to modify weights of this Network, for example, I would like to Multiply weights of first convolutional layer by 0.5, but I have received this error:
"You cannot set the read-only property 'Layers' of SeriesNetwork"
Is there any solution for this problem?
Thank you for advising me on this topic.
Reza.

请先登录，再回答此问题。

回答（4 个）

Peter Gadfort
Peter Gadfort 2018-6-8
If you save the network to a struct, you can edit it there and then load it back into the network (this works in 2018a, I have not tested in any other version yet)
tmp_net = convnet.saveobj;
tmp_net.Layers(2).Weights = 0.5 * tmp_net.Layers(2).Weights;
convnet = convnet.loadobj(tmp_net);
classify(convnet, testDigitData);
This will allow you to edit the Weights and Biases without retraining the network.
Carlo Tomasi
Carlo Tomasi 2018-2-21
While the Layers property of a SeriesNetwork is read-only, the weights and biases of a net.cnn.layer.Layer can be set at will. Thus, one way to address your problem is to copy all the weights from convnet.Layers to layers, modify the weights of layers as you wish, and then make a new convnet with trainNetwork. For instance:
for l = 1:length(layers)
if isprop(layers(l), 'Weights') % Does layer l have weights?
layers(l).Weights = convnet.Layers(l).Weights;
end
if isprop(layers(l), 'Bias')% Does layer l have biases?
layers(l).Bias = convnet.Layers(l).Bias;
end
end
layers(2).Weights = 0.5 * layers(2).Weights; % Modify the weights in the second layer
convnet = trainNetwork(trainDigitData,layers,options); % Make a new network and keep training
Of course, this code assumes that the structures of layers and convnet.Layers match. I hope this helps.
  4 个评论
显示 2更早的评论
Kirill Korotaev
Kirill Korotaev 2018-4-25
Hello Jan, I am now trying to implement 'node KO' treatment, should I do something else except simply setting bias to 0? I have no effect on classification accuracy even when I set to 0 every bias in 1st convlayer, probably I am doing it wrong
Theron FARRELL
Theron FARRELL 2019-12-3
I think that Kirill may set WeightLearnRateFactor and BiasLearnRateFactor of corresponding layers 0 so that parameters will not be updated through training.
As for Jan Jaap van Assen 's issue, I suppoe that you may try a workaround by adjusting the traning epochs to a point at which you wish to manipulate those paramters, save and reload your network at that point, specify parameters manually, for example
% W and b can be user-specified
layer = convolution2dLayer(filterSize,numFilters, ...
'Weights',W, ...
'Bias',b)
then start training again...

请先登录，再进行评论。

Jan Jaap van Assen
Jan Jaap van Assen 2018-4-23
I was a bit annoyed by this very basic restriction where we can't manually set weights. Using Carlo Tomasi's answer as inspiration I wrote a function that allows you to replace weights for layers of DAGNetworks. Sadly, it is required to call trainNetwork, which massively slows down the process. It only changes the weights you specify, all the other weights stay unchanged in the new net object.
For example, to set the weights of layer 5 all to zero:
newNet = replaceWeights(oldNet,5,zeros(size(oldNet.Layers(5).Weights)));
See the code below, I also attached the function file. I hope somebody can correct me with faster solutions.
function newNet = replaceWeights(oldNet,layerID,newWeights)
%REPLACEWEIGHTS Replace layer weights of DAGNetwork
% newNet = replaceWeights(oldNet,layerID,newWeights)
% oldNet = the DAGnetwork you want to replace weights.
% layerID = the layer number of which you want to replace the weights.
% newWeights = the matrix with the replacement weights. This should be
% the original weights size.
% Split up layers and connections
oldLgraph = layerGraph(oldNet);
layers = oldLgraph.Layers;
connections = oldLgraph.Connections;
% Set new weights
layers(layerID).Weights = newWeights;
% Freeze weights, from the Matlab transfer learning example
for ii = 1:size(layers,1)
props = properties(layers(ii));
for p = 1:numel(props)
propName = props{p};
if ~isempty(regexp(propName, 'LearnRateFactor$', 'once'))
layers(ii).(propName) = 0;
end
end
end
% Build new lgraph, from the Matlab transfer learning example
newLgraph = layerGraph();
for i = 1:numel(layers)
newLgraph = addLayers(newLgraph,layers(i));
end
for c = 1:size(connections,1)
newLgraph = connectLayers(newLgraph,connections.Source{c},connections.Destination{c});
end
% Very basic options
options = trainingOptions('sgdm','MaxEpochs', 1);
% Note that you might need to change the label here depending on your
% network in my case '1' is a valid label.
newNet = trainNetwork(zeros(oldNet.Layers(1).InputSize),1,newLgraph,options);
end
Hakan
Hakan 2019-12-1
I would like ask another question. I am thinking it is related to this subject. I would like to make int8 Quantization to the weights. I read this article:
https://www.mathworks.com/company/newsletters/articles/what-is-int8-quantization-and-why-is-it-popular-for-deep-neural-networks.html?s_eid=psm_dl&source=15308
I used Carlo Tomasi method (one way to address your problem is to copy all the weights from convnet.Layers to layers, modify the weights of layers as you wish) And used fi function to make the 8 bit Quantization.
and then tried to make changes as fallows:
layers(2).Weights(2) = fi(layers(2).Weights(2),1,8);
I tried just for one weight to see if it changes from single to int8. The simple code above works with no error but the weight type is still single. Is there any way to make the wieght int8?
  3 个评论
显示 1更早的评论
David Haas
David Haas 2021-3-9
Hello Hakan,
The int8-quantization Technical article you linked to states the following:
However, the execution time on the discovery board shows that the single-precision variant takes an average of 14.5 milliseconds (around 69 fps) to run while the scaled version is a little slower and takes an average of 19.8 milliseconds (around 50 fps). This might be because of the overhead of the casts to single precision, as we are still doing the computations in single precision (Figure 7).
So basically, it is an 8-bit accurate number and it is cast back into single precision.
It is not clear to me why you need it to be a true 8 bit integer for evaluating the quantization as long as it has equivalent 8-bit accuracy for evaluation.
Ali Al-Saegh
Ali Al-Saegh 2021-3-10
Hello David,
The purpose of using int8 quantization method is to reduce the memory requirement of the neural network, which can be up to 75% of its original size. However, the obtained network after quantization shows the same size as the original one. If the memory size is not reduced and more time is required for the inference process, so what is the benefit of quantization?
I really appreciate your help in giving clarification for that.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Parallel and Cloud 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by