Ridge regression coefficient question

Question

Jason 2012-2-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/29373-ridge-regression-coefficient-question

编辑： George 2013-10-16

I'm confused about how ridge regression coefficients are generated in matlab. Any help would be appreciated. An example of the issue is shown below.

Thanks,

JG

N = 200;
p = 30;
y = rand(N,1);
X = [ones(N,1),rand(N,p)];
lambda = 1;
R = X'*X + lambda*eye(size(X,2));
Rinv = inv(R);
b_ridge = Rinv*X'*y;
y_ridge = X*b_ridge;
XX = X(:,2:end);
b_ridge_matlab = ridge(y,XX,lambda,0);
y_ridge_matlab = X*b_ridge_matlab;
% why are b_ridge and b_ridge_matlab different?  I thought that
%the 0 option in ridge eliminated all scaling and was useful for
%prediction (i.e., y_pred = X_new*b).

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Tom Lane 2012-2-17

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/29373-ridge-regression-coefficient-question#answer_37747

在 MATLAB Online 中打开

Good question! This took a while to figure out, and I can see the help text is not clear about it. The calculations are actually always based on a scaled X under the hood, but the results are adjusted later to be usable with the unscaled data. In particular, the ridge parameter is interpreted as applying to the scaled data. You can reproduce the ridge results by computing R in your code as follows:

R = X'*X + lambda*diag(var(X));

3 个评论
显示 1更早的评论隐藏 1更早的评论

Tom Lane 2012-2-17

I agree the help text is confusing. The definition you quote is accurate when X is scaled. I think the alternative with the "0" flag ought to be described as presenting the ridge coefficients, computed the same way, but then post-processed so they can be used with the original X variables. Unless I misunderstand, they do serve that purpose. Try changing your script to include a real relationship between X and y, and at the end plot the fitted and observed values:

y = X*(5./(1:31)')+rand(N,1);

...

scatter(y_ridge_matlab,y)

Jason 2012-2-19