error message using polyfit (nonlinear regression)

Question

Locks 2013-4-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/73028-error-message-using-polyfit-nonlinear-regression

hi,

I get the following error meesage using the polyfit function:

Warning: Polynomial is badly conditioned. Add points with distinct X
         values, reduce the degree of the polynomial, or try centering
         and scaling as described in HELP POLYFIT.

Has anybody see that before and has an idea what I need to do? I tried it with the help function, but I didn't understand what excatly could be false

the code I am using if the following, in case it helps:

   if length(dataT(:,1))==1
              SlopeSkew(number)=0;
              elseif length(dataT(:,1))==2
              SlopeSkew(number)=0;
              else
              % x is the Strike
              x= dataT(:,2);
              %is the implied volatility
              y=dataT(:,10);
              p = polyfit(x,y,2);
              f = polyval(p,x);
  thanks!
              a=p(3);
              b=p(2);
              c=p(1);
              SlopeSkew(number)=b+2*c.*x;
              Slope=SlopeSkew';

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Image Analyst 2013-4-21

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/73028-error-message-using-polyfit-nonlinear-regression#answer_83007

So what is the length of x and y, and do you have any repeated x values?

4 个评论
显示 2更早的评论隐藏 2更早的评论

Locks 2013-4-21

在 MATLAB Online 中打开

yes, I have repeated x and y values

the full code looks like this:

for putCall =1:2
    dataPutCall = data16(data16(:,3)==putCall,:);
    dates=unique(dataPutCall(:,6));
    for i=1:length(dates)
        date=dates(i);
          dataDate = dataPutCall(dataPutCall(:,6) == date,:);
          Ts = unique(dataDate(:,5));
          for indexT = 1:length(Ts)
              T = Ts(indexT);
              dataT = dataDate(dataDate(:,5) == T,:);
              number=dataT(:,13);
              if length(dataT(:,1))==1
              SlopeSkew(number)=0;
              elseif length(dataT(:,1))==2
              SlopeSkew(number)=0;
              else
              % x is the Strike
              x= dataT(:,2);
              %is the implied volatility
              y=dataT(:,10);
              p = polyfit(x,y,2);
              f = polyval(p,x);
              a=p(3);
              b=p(2);
              c=p(1);
              SlopeSkew(number)=b+2*c.*x;
              Slope=SlopeSkew';
              end
          end
      end
  end

so there is not a specific lenght of x, because x is for each iteration different and from the error message I can't see, which x is causing a problem.

do you know what the error message stands for?

Image Analyst 2013-4-21

在 MATLAB Online 中打开

If you have two Y values for the same x value, then it doesn't like that and will complain. All your x values have to be unique. You can use

uniqueX = unique(x)

and see if length(uniqueX) is the same length as length(x). If they're the same then there are no repeats. If unique() returns a shorter vector, then at least one x value is repeated and you need to decide how to handle that. You might be able to just add a very small amount to one of the x's, like 0.000001, just to make sure they are not the same anymore.

请先登录，再进行评论。

Answer 2

Tom Lane 2013-4-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/73028-error-message-using-polyfit-nonlinear-regression#answer_83135

在 MATLAB Online 中打开

There was a time when this function issued an error asking you not to have repeated X values. But the new error message is more accurate. You don't need unique X values. It's just that repeated X values won't allow you to estimate higher-order polynomials. So for instance:

x = [1;2;3;3;4];
y = (1:5)';
polyfit(x,y,2)
polyfit(x,y,4)

The first call to polyfit works. The second would work if we had 5 points with distinct X values, but it doesn't work here because the 4 distinct X values allow polynomials only up to an exponent of 3.

In your example of fitting up to power 2, it seems like you either don't have 3 distinct points, or you have very ill-conditioned data.

2 个评论
显示无隐藏无

Locks 2013-4-22

在 MATLAB Online 中打开

I tried it first for a smaller sample and there I had no issues doing that, but for the larger sample the error message appeared. This part of the code here:

                if length(dataT(:,1))==1
                SlopeSkew(number)=0;
                elseif length(dataT(:,1))==2
                SlopeSkew(number)=0;
                else

should make sure that for the computation, there should be always at least 3 datapoints, so do you have any idea what else it could be

Tom Lane 2013-4-23

在 MATLAB Online 中打开

I don't know exactly. Of the following three, the first one works. The second does not because there are only two distinct x values. The third does not because it is very ill-conditioned.

polyfit([0;1;2],[10;20;30],2);
polyfit([0;1;1],[10;20;30],2);
polyfit([0;eps;1],[10;20;30],2);

I don't know what the issue is in your case. Try to boil this down to a specific call to polyfit, then examine the x and y values in that call and see how they look.

请先登录，再进行评论。

Answer 3

Jan 2013-4-23

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/73028-error-message-using-polyfit-nonlinear-regression#answer_83222

在 MATLAB Online 中打开

Instead of repeated values, did you test the condition of the problem already? The docs suggest to use

[p, S, mu] = polyfit(x,y,n)

for a proper scaling. The matrix for the least-squares fit is ill-conditioned, when the values of x have a wide range and are far away from zero. Therefore the scaling does:

xx = (x - mean(x)) / std(x)

to get all data near to zero. The conversion back to the original values in POLIVAL is trivial.

3 个评论
显示 1更早的评论隐藏 1更早的评论

Locks 2013-4-23

sorry but I am not sure what excatly I need to change. I looked at the docs youre mentionning http://www.mathworks.ch/ch/help/matlab/ref/polyfit.html before startig with the code and p = polyfit(x,y,n) should work as well or am I mistaken? a bit strage is, that everything is working fine for different datasets with each 8000 rows, but for a dataset that consists of 30'000 rows it's not worwing. I am tring to find out at which point the problem is but it would be really heplful to get a hint what I need to focus at.

Locks 2013-4-24