Why I get two different covariance matrix?

Question

ali yaman 2022-7-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1759520-why-i-get-two-different-covariance-matrix

评论： ali yaman 2022-7-14

采纳的回答： Paul

在 MATLAB Online 中打开

Hi all,

Prof. Andrew Ng in his ML class says that we can calculate covariance matrix as: 1/m*X'*X .

Where;

examples are in rows of X,

X' is transpose of X,

and, m is number of examples.

For example:

X=randi(12,[6,2]);
cov1=1/size(X,1)*X'*X
cov1 = 2×2
   92.5000   42.0000
   42.0000   21.8333

And, covariance with cov function is:

cov2=cov(X)
cov2 = 2×2
    2.7000   -0.9000
   -0.9000    1.9000

As you can see, cov1 is different from cov2 !!!

What is the reasan for that? Do you have any idea?

Thanks

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Paul 2022-7-14

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1759520-why-i-get-two-different-covariance-matrix#answer_1006980

在 MATLAB Online 中打开

Hi Ali,

Perhaps Prof. Ng has some additional assumptions about the data that aren't included in your question. To compute the covariance we have to subtract off the mean. As for whether or not the outer product should be scaled by 1/m or 1/(m-1) depends on assumptions about the underlying data. IIRC, if we know the data is drawn from a Normal distribution then divide by m (perhaps also for other distributions as well?), but typically we don't assume that and so divide by m-1 for unbiased estimation. As can be seen below, cov subtracts the mean and divides by m-1.

rng(100);
X=randi(12,[6,2])
X = 6×2
     7     9
     4    10
     6     2
    11     7
     1    11
     2     3
cov1=1/(size(X,1)-1)*(X-mean(X))'*(X - mean(X))
cov1 = 2×2
   13.3667   -1.6000
   -1.6000   14.0000
cov(X)
ans = 2×2
   13.3667   -1.6000
   -1.6000   14.0000

As for the second question

sigma=[6 2;2 3]; % cov matrix
[a1,v]   = eig(sigma)
a1 = 2×2
    0.4472   -0.8944
   -0.8944   -0.4472
v = 2×2
    2.0000         0
         0    7.0000
[a2,s,~] = svd(sigma)
a2 = 2×2
   -0.8944   -0.4472
   -0.4472    0.8944
s = 2×2
     7     0
     0     2

we see that eig and svd just have a different order for the results.

3 个评论
显示 1更早的评论隐藏 1更早的评论

Paul 2022-7-14

Which one of eig() or svd() to use? it would never occur to me to use svd() to get the eigenvectors of a symmetric matrix. Don't know if eig() or svd() is better for that special case. Asking that as a new question is more likely to get the attention of knowledgeable people that can answer.

ali yaman 2022-7-14

@Paul Ok, Thanks Bro.

请先登录，再进行评论。

Answer 2

ali yaman 2022-7-14

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1759520-why-i-get-two-different-covariance-matrix#answer_1006905

在 MATLAB Online 中打开

What about the second question!!

Just like covarince matrixe we can get two different eigenvectors.

For example;

sigma=[6 2;2 3]; % cov matrix
[a,~]=eig(sigma)
a = 2×2
    0.4472   -0.8944
   -0.8944   -0.4472

And, lets calculate eigenvectors with svd, singular value decomposition function:

[a,~,~]=svd(sigma)
a = 2×2
   -0.8944   -0.4472
   -0.4472    0.8944

As you can see we are getting two different eigenvectors value.

What is the reason?

Thanks, in advance.

4 个评论
显示 2更早的评论隐藏 2更早的评论

John D'Errico 2022-7-14

编辑：John D'Errico 2022-7-14

Since this is probably of some general interest, I'll actually post a question of my own, then answer it myself, discussing the relative issues between eig and svd.

https://www.mathworks.com/matlabcentral/answers/1760195-should-i-use-eig-or-svd-to-compute-the-eigenvalues-of-a-symmetric-matrix

ali yaman 2022-7-14