How do i quantitatively evaluate Prediction Interval accuracy?

Question

israt fatema 2023-2-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1913425-how-do-i-quantitatively-evaluate-prediction-interval-accuracy

回答： Kartik 2023-4-18

I am using a time series data using GPR Model https://au.mathworks.com/help/stats/gaussian-process-regression-models.html and then need to quantitatively evaluated Prediction Intervals accuracy with PICP (PI coverage probability) and PINAW (PI normalized average width) for assessing the accuracy of the PIs. I need help with the writing matlab code for PICP and PINAW. TIA

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Kartik 2023-4-18

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1913425-how-do-i-quantitatively-evaluate-prediction-interval-accuracy#answer_1217878

Hi,

To evaluate the accuracy of Prediction Intervals (PIs) using PICP and PINAW in MATLAB, you can follow these steps:

Load or generate your time series data and split it into training and testing sets.
Train your Gaussian Process Regression (GPR) model on the training data using the fitrgp function.
Use the predict function to generate mean predictions and standard deviations for the testing data.
Compute the lower and upper bounds of the PIs for each prediction by adding and subtracting a multiple of the standard deviation from the mean prediction.
Calculate the PICP by dividing the number of actual observations within the PIs by the total number of observations. The PICP should ideally be close to the specified confidence level of the PIs (e.g., 95%).
Compute the PINAW by averaging the widths of the PIs and dividing by the range of the data. The PINAW should ideally be small, indicating narrow and precise PIs.

Refer to the following MathWorks documentation for more info:

https://www.mathworks.com/help/stats/gaussian-process-regression-models.html#mw_e2ac9ebf-6dc7-4b5c-947e-2c146fb2423d