Ridge regression coefficient question
显示 更早的评论
I'm confused about how ridge regression coefficients are generated in matlab. Any help would be appreciated. An example of the issue is shown below.
Thanks,
JG
N = 200;
p = 30;
y = rand(N,1);
X = [ones(N,1),rand(N,p)];
lambda = 1;
R = X'*X + lambda*eye(size(X,2));
Rinv = inv(R);
b_ridge = Rinv*X'*y;
y_ridge = X*b_ridge;
XX = X(:,2:end);
b_ridge_matlab = ridge(y,XX,lambda,0);
y_ridge_matlab = X*b_ridge_matlab;
% why are b_ridge and b_ridge_matlab different? I thought that
%the 0 option in ridge eliminated all scaling and was useful for
%prediction (i.e., y_pred = X_new*b).
采纳的回答
Tom Lane
2012-2-17
Good question! This took a while to figure out, and I can see the help text is not clear about it. The calculations are actually always based on a scaled X under the hood, but the results are adjusted later to be usable with the unscaled data. In particular, the ridge parameter is interpreted as applying to the scaled data. You can reproduce the ridge results by computing R in your code as follows:
R = X'*X + lambda*diag(var(X));
3 个评论
Jason
2012-2-17
Tom Lane
2012-2-17
I agree the help text is confusing. The definition you quote is accurate when X is scaled. I think the alternative with the "0" flag ought to be described as presenting the ridge coefficients, computed the same way, but then post-processed so they can be used with the original X variables. Unless I misunderstand, they do serve that purpose. Try changing your script to include a real relationship between X and y, and at the end plot the fitted and observed values:
y = X*(5./(1:31)')+rand(N,1);
...
scatter(y_ridge_matlab,y)
Jason
2012-2-19
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Linear Regression 的更多信息
Translated by 
