NARX - a strange thing happens

1 次查看(过去 30 天)
Giuseppe Menga
Giuseppe Menga 2022-7-16
回答: Krishna 2023-11-27
I control a lower limb exoskeleton for performing the sit-to-stand exercise using electromiographical signals with a NARX net.
With the standard procedure, I set some of the input samples for training, for validation and for test, starting with open loop and then closing the loop.
Casually, using "net.divideFcn = 'divideind'" I set all input samples for training without validation and test.; Obiously, starting with the same random numbers the final performance is better without validation and test than in the other case, as can be seen in the first two figures obtained in the maglev example.
Then I saved the net parameters for the case of no validation and test and with those parameters I started directly the training in closed loop with identical samples for training, validation and test, as before. As can be seen in the third figure the performace is close to the training without validation and test.
Conclusion: the net parameters for the case of not validation and test are the best also when validation and test are performed.
What is wrong in my reasoning?
Giuseppe

回答(1 个)

Krishna
Krishna 2023-11-27
Hello Giuseppe,
The issue you're encountering is known as overfitting in machine learning. Essentially, by using all available data for training the networks, the model may fit the data very well, possibly too well. Consequently, when testing the model with the same data it was trained on, it will likely perform well because it's already familiar with that data. However, when presented with new data, the model's performance will likely be worse than before because instead of learning general patterns, it has simply memorized the dataset, resulting in overfitting. That is why it is recommended to have 3 sets of dataset training, validation, and testing.
Please go through this documentation to know the importance of these 3 datasets,
Hope this helps.

类别

Help CenterFile Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by