K-Fold validation to compute bias correction.

Question

Ashfaq Ahmed 2024-2-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2085983-k-fold-validation-to-compute-bias-correction

回答： Chetan 2024-3-5

Hi all!

Say, I have an in situ buoy data set in a variable called Tb. Now, I have corresponding satellite measurements for each buoy data, and the data are stored in the Ts variable. So, both Tb and Ts are of the same size.

Now, I am trying to calculate the error E. Where E = Tb - Ts -b.

Here, b = bias. This bias has two components, and we assume that b = m*Ts + n (m = temperature dependent bias component and n = the shift in the mean)

So, ultimately, E = Tb - Ts - m*Ts - n.

I want to use the K-fold validation operation to get the values for m and n.

Can anyone please suggest me some solution for it? Any feedback will be greatly appreciated!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Chetan 2024-3-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2085983-k-fold-validation-to-compute-bias-correction#answer_1421126

在 MATLAB Online 中打开

Hi @Ashfaq Ahmed,

t seems like you are trying to do k- fold validation.To use K-Fold validation for bias correction with the given, follow these steps:

1. Prepare Data: Align and size-match your buoy "Tb" and satellite "Ts" data

2. Set Up K-Fold: Choose a number of folds, typically 5 or 10, based on your data size.

3. Split Data: Divide data into "K" non-overlapping folds.

Validate in K-Folds:
Treat one fold as validation and the rest as training.
Use linear regression on the training set to model "Tb" from "Ts" accounting for bias.
Estimate m and n to minimize E = Tb - Ts - m*Ts - n.
Optionally, validate these estimates using the validation set.

4. Average Results: Calculate the mean of m and n estimates from all folds for final values.

Here is the MATLAB Code Sample for the same:

% Assuming Tb and Ts data are defined
K = 5; % Example fold number
cv = cvpartition(length(Tb), 'KFold', K);
m_values = zeros(K, 1);
n_values = zeros(K, 1);
for i = 1:K
    trainIdx = training(cv, i);
    model = fitlm(Ts(trainIdx), Tb(trainIdx), 'Intercept', true);
    m_values(i) = model.Coefficients.Estimate(2); % m estimate
    n_values(i) = model.Coefficients.Estimate(1); % n estimate
end
m_final = mean(m_values);
n_final = mean(n_values);
disp(['Estimated m: ', num2str(m_final)]);
disp(['Estimated n: ', num2str(n_final)]);