how do i solve least squares problems with newtons method?

13 次查看（过去 30 天）
apostolos georgantopoulos
评论： Danae Katsanta 2018-12-13
I was wondering if it is possible to solve least squares problems using matlab for 100% of the way. What i mean:
I wrote this code for finding the variables of the polynomial f(x)=a*x+b that minimize the sum of (yi-f(xi))^2 for the given x,y.
x=linspace(-5, 5,100);
y = [zeros(1,50) + 0.2*rand(1,50), ones(1,50)-0.2*rand(1,50)];
sol=[1;1];
sumx=0;
sumy=0;
sumxsq=0;
sumysq=0;
sumxy=0;
for i=1:100
sumx=sumx+x(i);
sumy=sumy+y(i);
sumxsq=sumxsq+x(i).^2;
sumysq=sumysq+y(i).^2;
sumxy=sumxy+x(i).*y(i);
F=[sol(1)*sumxsq+sol(2)*sumx-sumxy; sol(1)*sumx+101*sol(2)-sumy];
J=[sumxsq,sumx;sumx,101];
sol=sol-J\F;
endfor
sol
As you can see, I calculated F and J for the Newton-Raphson method on paper using the theory of least squares. But there is a problem with that, as it requires you to solve a lot of things on paper and it changes for each starting f(x) that you want to fit. I tried doing the same thing for a starting f that is given by: s(z)=1/(1+exp(-z)), f(x)=s(a*x+b), and i had trouble with the complexity of the equation as i couldnt find F and J easily. So, is there anything i missed in this example and its actually easy to calculate, or can i use newtons method in another way to solve this problem without calculating anything on paper. Thanks in advance and sorry of the long post!
  5 个评论
显示 3更早的评论
apostolos georgantopoulos
apostolos georgantopoulos 2018-12-13
It was specified that it should be solved using Newton-Raphson method (and later also the altered Newton-Raphson where you keep the Jacobian stable) on the book that i found this specific problem in and there are no similar examples. I don't mean to be too needy or anything, I'm just really baffled on how to solve this with this particular function whose derivatives are so complicated. Appreciate any info.
Danae Katsanta
Danae Katsanta 2018-12-13
I had to solve the excact problem. I found this one:
function s = jacobian(f, x, tol)
% f is a multivariable function handle, x is a starting point
if nargin == 2
tol = 10^(-5);
end
while 1
% if x and f(x) are row vectors, we need transpose operations here
y = x' - jacob(f, x)\f(x)'; % get the next point
if norm(f(y))<tol % check error tolerate
s = y';
return;
end
x = y';
end
function j = jacob(f, x) % approximately calculate Jacobian matrix
k = length(x);
j = zeros(k, k);
for m = 1: k
x2 = x;
x2(m) =x(m)+0.001;
j(m, :) = 1000*(f(x2)-f(x)); % partial derivatives in m-th row
end

请先登录，再进行评论。

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Newton-Raphson Method 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by