Can I obtain the covariance matrix of a stochastic process with plenty of measurements?

Question

Jaime De La Mota Sanchis 2021-6-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/857865-can-i-obtain-the-covariance-matrix-of-a-stochastic-process-with-plenty-of-measurements

回答： Bjorn Gustavsson 2021-6-25

在 MATLAB Online 中打开

Hello everyone.

I have implemented the Karhunen-Loève expansion as per this question.

I have tested it with samples of data obtaining possitive results.

However, now I am dealing with a matrix of dimensions 211302*50, meaning that I have roughly 200000 observations of 50 random variables.

If I try to calculate the covariance matrix of these using the observations as columns (as per the previus link), the program crashes and the error returned is:

Error using  * 
Requested 211302x211302 (332.7GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take
a long time and cause MATLAB to become unresponsive.
Error in cov (line 155)
c = (xc' * xc) ./ denom;
Related documentation

Is there a way to do what I want or is the matrix just too big?

Best regards.

Jaime.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Sharmin Kibria 2021-6-25

As the error suggested, the covariance matrix you are trying to build is too big. Your solution needed to allocate memory that was larger than what is allowed for array storage. That is why MATLAB crashed.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Bjorn Gustavsson 2021-6-25

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/857865-can-i-obtain-the-covariance-matrix-of-a-stochastic-process-with-plenty-of-measurements#answer_733865

在 MATLAB Online 中打开

To me it seems that you have misunderstood the dimensions. Your covariance-matrix should, to my understanding be 50x50 when you have 50 random variables observed 200000 times. With a call like this:

tic,C = cov(randn(211302,50));toc

I get a 50-by-50 covariance-matrix C in ~0.22 s.

HTH