Something wrong in fitrlinear with ridge regularization?

2022 3 28
1 个回答
回答已采纳
更新时间：2023 12 31
17 次查看（30 天）

0 个投票

There is something wrong in the attached code. I couldn't figure out where it went wrong. Any help is highly appreciated!

5 个评论
显示 3更早的评论 隐藏 3更早的评论

Peter He
Peter He 2022-3-31
Hi Prince,
Thank you for looking into this.
There is no syntax error but rather a semantic error maybe. I explained the error in the live script - the result of setting "Lambda" to 0 for ridge regression should return a model exactly the same as the linear model w/o regularization/shrinkage. But I get a much worse model instead. Here is the line that I tested the above point in the attached code:
mdl2=fitrlinear(X(idxTrain,:),y(idxTrain),'Learner','leastsquares','Regularization','ridge','Lambda',0);
I got mse = 54.3139
if I use the following:
mdl = fitlm(X(idxTrain,:),y(idxTrain));
I got mse = 16.8294, which is significantly better. I also plotted the observed vs. fitted, which also shows very poor fitting for ridge regression.
Peter He
Peter He 2022-3-31
I'm not sure if there is any scaling involved for the predictors - I couldn't find any information from the document of this function. But it seems that the response is not scaled. Just a potential issue I have thought about but haven't tested.
Biral Pradhan
Biral Pradhan 2022-9-28
The "fitrlinear" method uses iterative solvers that converge subject to tolerances on beta coefficients and gradient. In this case, optimization doesn't occur because the first step does not move the beta estimates away from the initial zero values far enough. Kindly set the "BetaTolerance" parameter to a lower value, say 1e-6 in order to resolve this issue. For more information, kindly follow the below documentation link:
https://in.mathworks.com/help/stats/fitrlinear.html#d124e418872
I would also recommend setting "Verbose" to 1 in order to obtain more information throughout the optimization process.
Peter He
Peter He 2023-12-31
Thank you Biral. That indeed was the problem. Here is a link to a related question that I asked and got a similar answer to it. But I was able to verify the problem using that example.
https://www.mathworks.com/matlabcentral/answers/1689650-another-example-where-fitrlinear-performs-badly-and-i-don-t-know-where-the-problem-is#answer_1380091
Thanks again.
Peter

请先登录，再进行评论。

请先登录，再回答此问题。

 采纳的回答

Peter He
Peter He 2023-12-31

0 个投票

Thank you Biral. That indeed was the problem. Here is a link to a related question that I asked and got a similar answer to it. But I was able to verify the problem using that example.
https://www.mathworks.com/matlabcentral/answers/1689650-another-example-where-fitrlinear-performs-badly-and-i-don-t-know-where-the-problem-is#answer_1380091
Thanks again.
Peter

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Support Vector Machine Regression 的更多信息

产品

版本

R2020a

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by