Regularized logistic regression - Gradient calculation

Question

Nik 2016-6-26

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/292318-regularized-logistic-regression-gradient-calculation

回答： Xinwei LONG 2020-2-18

Hello, I am doing a regularized logistic regression task and stuck with the partial derivatives. The gradient should be normalized (added lambda/m*theta), except for the first term theta(1). So, I had the following code, which works incorrectly:

grad(1) = 1/m*((sigmoid(X(:,1)*theta(1))-y)'*X(:,1));
grad(2:end) = 1/m*((sigmoid(X(:,2:end)*theta(2:end))-y)'*X(:,2:end))' + lambda/m*theta(2:end);

Finally, I came to another solution, which works fine:

grad = (1/m*(sigmoid(X*theta)-y)'*X)';
temp = theta;
temp(1) =0;
grad = grad + lambda/m*temp;

Can someone please explain, why the first option is incorrect. Thanks a lot!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Xinwei LONG 2020-2-18

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/292318-regularized-logistic-regression-gradient-calculation#answer_416044

Hi,

I initially wrote the same form of vectoriztion in dealing with cost function and gradients.

Here's what I found out the right answers:

grad(1)=(1/m)*sum(((sigmoid(X*theta)-y).*X(:,1)),1);

grad(2:end)=(1/m)*sum(((sigmoid(X*theta)-y).*X(:,2:end)),1)'+(lambda/m)*theta(2:end);

Please find the differences in inputs of sigmoid function. The gradient equation for theta_0 and other thetas still required the same input in sigmoid function.