How to "prune" a one hidden layer NN such that the off diagonal elements of the weights, that is, "net.IW{1}" and "net.LW{2}", are zeros?

Question

Shashi Kant 2020-5-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/531198-how-to-prune-a-one-hidden-layer-nn-such-that-the-off-diagonal-elements-of-the-weights-that-is

回答： Abdelwahab Afifi 2021-4-12

How can I create a single hidden layered NN such that the off main diagonal elements of the weights, that is, "net.IW{1}" and "net.LW{2}", are zeros?

Please see my partial script below with a question there. Can someone help me with this toy example?

%% Create a dataset with a network
[x,t]    = crab_dataset; %load the dataset
x        = x(1:2,:); % Input length 
net      = patternnet(2); %create the network
net      = configure(net,x,t);
view(net);
w1 = net.IW{1}; %the input-to-hidden layer weights (but w1 
w2 = net.LW{2}; %the hidden-to-output layer weights
b1 = net.b{1}; %the input-to-hidden layer bias
b2 = net.b{2}; %the hidden-to-output layer bias
%% QUESTION (or INTENTION): 
% How to setup the net such that the off diagonal elements of w1 or net.IW{1}  and w2 or net.LW{2} are ZEROs after training the network?
%% then, I would like to train the network with my defined network!
% [net,tr] = train(net,x,t); %train the network

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Srivardhan Gadila 2020-5-29

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/531198-how-to-prune-a-one-hidden-layer-nn-such-that-the-off-diagonal-elements-of-the-weights-that-is#answer_442411

As per my knowledge w.r.t shallow nerual networks you cannot freeze non-diagonal weights & make the diagonal wieghts only to update since the property net.layerWeights{i,j}.learn is defined for the entire connections between layers i and j.

I would suggest you to use Deep Nerual networks instead of shallow nerual networks & define custom Deep Learning layer to achieve your functionality. Refer to Define Custom Deep Learning Layers & Define Custom Deep Learning Layer with Learnable Parameters

Other suggestion w.r.t shallow nerual networks approach: (may or may not be useful)

Set the net.trainParam.epochs to 1 & place the [net,tr] = train(net,x,t); in a for loop iterated over total number of epochs, then after each epoch set the non-diagonal weights to zero.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Shashi Kant 2020-5-29

编辑：Shashi Kant 2020-5-29

@Srivardhan Gadila: Thank you for your comment. Would it be possible for you to create a simple example with the DNN toolbox? I would greatly appreciate it.

请先登录，再进行评论。