Neural Network (NARX) performance interpretation - how small is small?

2 次查看(过去 30 天)
I have built a neural network to forecast currency pair prices (trend) a few steps ahead. How do I interpret the performance plot results? For instance, the error histogram, the NMSE (what is considered small for currency pairs? How do I assess if these metrics are optimal?

回答(1 个)

Greg Heath
Greg Heath 2017-8-19
If you naively model a function with a straight line, the MSE is just the variance of the function and varies with the scale of the function.
In previous MATLAB posts I have used the notations
MSE00 = vart1 = mean(var(target'),1)
and use it as a standard for measuring MSE.
Using this NAIVE model as a comparative standard, the normalized MSE is defined by
NMSE = MSE/mean(var(target',1))
which is related to the statistical RSQUARE via
RSQUARE = 1 - NMSE
For reasonable models both NMSE and RSQUARE lie within the closed interval [ 0,1].
For most regression and classification models, I use a goal
NMSEgoal = 0.01 % Regression & Classification
However, in order to obtain that goal for CLOSED LOOP time series, I use the OPEN LOOP goals
NMSEgoal = 0.005 or 0.001
I don't know of anyone who complains when Rsquare >= 0.99 or equivalently, NMSE <= 0.01.
Hope this helps.
Thank you for formally accepting my answer
Greg

类别

Help CenterFile Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by