You have to consider that you are looking at samples from a sampling distribution of a statistic, the sample covariance.
Consider the following.
I'll set the random number generator to its default settings for reproducible results.
rng default;
X = randn(100,2);
plot(X(:,1),X(:,2),'.')
You can see in the scatter plot that there is no obvious relationship between the two columns. The scatter plot is very much what you expect for uncorrelated random variables. Now, execute:
The off diagonal covariance is 0.0881. The variances of the two random variables are 1.3512 and 1.010, close to the expected value of 1, but not exact.
If you have the Statistics Toolbox, you can use mvnrnd() to model correlation.
mu = [0 0]; Sigma = [1 .6; .6 1];
X = mvnrnd(mu,Sigma,100);
plot(X(:,1),X(:,2),'.')
The scatter plot shown above shows a clear linear relationship between the two random variables.
Now the off diagonal elements of the matrix are close to 0.6.
Finally, to look at a histogram indicative of the sampling distribution for the uncorrelated case, let's repeat the experiment 500 times.
for nn = 1:500
X = randn(100,2);
Y = cov(X);
cov_values(nn) = Y(1,2);
end
hist(cov_values)
mean(cov_values)
You should see that the sampling distribution looks Gaussian with a mean that is very close to the theoretical value of 0.
So bottom line, for any given realization you cannot expect the off diagonal covariance to be zero, but in repeated sampling, the statistic (sample covariance) will have zero mean. That is always the case when dealing with statistics, you have to take into account the sampling distribution.
