The ‘functionLayer’ from MATLAB's Deep Learning Toolbox enables the implementation of activation functions such as ‘sine’ or cosine’. This function enables the definition of custom activation functions by passing a function handle that operates elementwise on the input data.
For example, a ‘sine’ activation function can be defined as follows using ‘functionLayer’:
sinActivation = functionLayer(@(X) sin(X), 'Name', 'sineActivation');
For ‘tanh’ MATLAB provides an inbuilt ‘tanhLayer’ function which can be directly used into the neural network.
The following resources provide more information on how activation functions can be implemented MATLAB:
- tanhLayer: https://www.mathworks.com/help/releases/R2024b/deeplearning/ref/nnet.cnn.layer.tanhlayer.html
- functionLayer: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.functionlayer.html
- Deep Learning Functions: https://www.mathworks.com/help/deeplearning/referencelist.html?type=function&listtype=cat&category=index&blocktype=all&capability=&s_tid=CRUX_lftnav
- Discussion on available activation functions: https://www.mathworks.com/matlabcentral/answers/1662520-matlab-activation-function-list
I hope this helps!
