Switching an existing batch trained NAR network to continuous incremental training with adapt?
4 次查看(过去 30 天)
显示 更早的评论
I have a NAR network that has been batch trained on a large set of historical data with Levenberg-Marquardt training. I have tweaked many of the settings so it has reasonably good performance when testing on old data and now I would like to implement the NET on a stream of continuously arriving new data and have it adjust itself according to the new data with incremental training.
From what I understand the adapt function in matlab is what I should be using for incremental training but there appears to be very limited examples on how to use it so I have a few questions. So first of all: Is it even possible to take an existing batch trained net with all of its weight settings and feedback delays and overall architecture and build on it with incremental training of new data or does the net have to be made from start with incremental training? And if it’s possible, how? The adapt documentation only shows a tiny example of adapt in use with a narx network, are there any other examples/tutorials on the subject?
The whole area of incremental training appears to be quite hidden in the neural network toolbox and it’s not shown as a training option in nnstart for instance. From what I understand the training procedure with incremental training is considerably slower compared to batch training so that’s why I would prefer to use the best out of two worlds – batch on historical data and incremental on newly arriving data. One concern I have about this would be that there will always be a gap in the time series when switching between historical and current data, how would that affect the net, if for example there is a one week gap between the historical and current data that the net is exposed to will the net perceive that as a violent change in the time series and potentially “learn” the false behavior which could ruin the net?
What I ultimately would like to achieve is a live prediction scenario where my NAR net predicts a few time steps ahead of a time series to the best of its ability and when the “correct” new data arrives it adapts on it and takes it into account when performing the next prediction. To achieve that do I need to constantly close the net, retrain it and make the prediction, create an open net again and account for the new data and repeat forever? That approach seems like a lot of hassle, is there a better way to do this?
Thanks for any advice.
0 个评论
采纳的回答
Greg Heath
2015-5-27
It doesn't make any difference whether the additional training is batch or adaptive:
If the new data does not contain the dominant characteristics of the original data, the net will forget the characteristics of the original data.
This is the aptly named phenomenon called "FORGETTING".
Therefore, if additional training is planned, you must include additional data that characterizes the dominant characteristics of the original data.
Hope this helps.
Thank you for formally accepting my answer
Greg
更多回答(1 个)
Greg Heath
2015-5-28
If the new data is just a time extension of the old data, I recommend either
1. Use batch training on all of the data with the weights
initialized to those found in the initial batch training.
2. Use adaptive training on all of the data with the weights
initialized to those found in the initial batch training.
Hope this helps.
Thank you for formally accepting my answer
Greg
2 个评论
Greg Heath
2015-8-30
After you close the loop you must test netc. If performance is bad then train netc initialized with the weights obtained from open loop training.
I don't know about trying to adapt to just one point. Don't forget you have lags to deal with. You don't want the computer to grind away changing weights to fit one point and neglecting the others.
I don't know how accurate this is, but my gut might churn significantly if I tried to adapt to new data that wasn't at least twice as long as the longest delay. Sound like something to investigate.
Hope this helps. Greg
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!