Few quick observations
First: There are a lot of different ways to bootstrap a regression model. Some of the more common examples include
Parametric residual bootstrap:
- Start by generating a regression model
- Calculate the variance of the residuals
- Using this information to generate new residuals
- Generate new observations by adding new residuals to YHat
Nonparametric residual bootstrap
- Start by generating a regression model
- Calculate the residuals
- Generate new observations by sampling with replacement from the set of residuals and adding these to YHat
Paired bootstrap
- Sample with replacement from the set of observations
Each of these techniques has its own advantages and disadvantages. (At the most basic level, there is a tradeoff between convergence speed and how restrictive your assumptions are about the data set) For basic information about bootstrapping regression models, I strongly recommend the following
Second: There is also the whole issue about how best to get this up and running in MATLAB
Here’s some simple code that will show you how to bootstrap SSE and R^2 using a parametric residual bootstrap. You can easily change this to a nonparametric residual bootstrap using the randsample function to sample with replacement.
X1 = 100 * rand(100,1);
X2 = 100 * rand(100,1);
Y = 2*X1 + 3*X2 + 10*randn(100,1);
[foo GoF] = fit([X1 X2],Y, 'poly11')
YHat = foo(X1,X2);
resid = Y - YHat;
Scaling = std(resid);
% Simple bootstrap example
N_Boot = 1000;
SSE = zeros(N_Boot,1);
R_Sqrd = zeros(N_Boot,1);
for i = 1:N_Boot
[foo_b , GoF_b] = fit([X1 X2],YHat + Scaling * randn(100,1), 'poly11');
SSE(i) = GoF_b.sse;
R_Sqrd(i) = GoF_b.rsquare;
end
mean(SSE)
std(SSE)
mean(R_Sqrd)
std(R_Sqrd)
