Custom Neural Network with learnable weights in activation function
显示 更早的评论
Hi,
I want to create a Custom Neural Network with learnable parameters in the activation function. The learnable parameters are the weights of layers, but the weights are also used inside the activation function in a certain nonlinear relation. Normally, the activation function takes the inputs multiplied with the weights ( e,g, y1=f1(X*W1 + b1)) but here I want to use weights directly inside in a certain nonlinear relation i.e. y1=g1(X*W1 + b1,W1).
Custom layers can be created as metioned here:
But the way I want to create a custom layer with a specific activation function is what i need to know. Any help will be highly appreciated.
AR
回答（1 个）
Kausthub
2023-9-7
Hi Ahmed Rehan,
I understand that you would like to have a custom activation function that has weights as an additional input i.e., instead of y1= f1(X*W1 + b1) you would like to have y1=g1(X*W1 + b1, W1).
You could leverage the “Define Custom Deep Learning Layer with Formatted Inputs” article which explains how to provide inputs to a custom layer.
I have attached a simple working snippet regarding the same for your reference. Major changes from the article you mentioned (https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layer.html) include:
- Adding weights as a parameter in the layer definition
function layer = customActivationLayer(weights, args)
- Passing weights when using the layer
customActivationLayer(Weights);
Please have a look at the attached code sample for better understanding.
Hope this helps and clarifies your queries regarding how to provide weights to activation functions using custom layers!
类别
在 帮助中心 和 File Exchange 中查找有关 Image Data Workflows 的更多信息
2022-9-16
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
