Adding Dropout to narxnet

Question

Andriy Artemyev 2021-3-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/780822-adding-dropout-to-narxnet

评论： Andriy Artemyev 2021-3-30

Greetings! I wanted to ask if/how it is possible to add a dropout layer to a narxnet to improve regularization. Unfortunately I could not find any information elsewhere.

I Have a narxnet that used the last 3 lags of a timeseries and an exogenous input to forecast the next timestep and I would like to introduce regularization measures to help with overfitting. Thanks in advance! My current code looks as follow:

forecast_horizon = 1;

neurons = [5 5];

delays = 3;

inputDelays = (1:delays);

feedbackDelays = (1:delays);

net=narxnet(inputDelays, feedbackDelays, neurons);

net.trainFcn='trainbr';

net.trainParam.epochs = 40;

net = removedelay(net,forecast_horizon);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Shashank Gupta 2021-3-29

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/780822-adding-dropout-to-narxnet#answer_661684

Hi Andriy,

There are some workaround to add a dropout in narxnet, you can add the dropout by defining a custom transfer function to one of the layer. Details on how to create a custom transfer function is show here in the link.

Another convenient way is to not use a shallow network but go for deep networks. There are some resources you can check, try this, once you implement this, you can simply add a dropout layer. I will prefer deep network way. It's more easy, reliable and conveninet to implement.

I hope this helps.

Cheers.