zscore of an array with NaN's?

39 次查看(过去 30 天)
Hi there!
What is the best way to take the zscore of this array while keeping NaNs in their place, and without simply indexing the part of the array where there aren't NaNs?
Thanks!
test =
NaN
NaN
25.0182
8.1078
0.7304
-8.4954
13.1454
31.7136
-7.2601
10.2586
16.4023
-55.0183
-53.4840
-4.5846
NaN

回答(4 个)

Mbvalentin
Mbvalentin 2016-12-21
What zscore function does is simply ensure your data has zero mean and std = 1.
Say you have a matrix X where rows are measures and cols are different features (which are measured in different scales, so in order to compare one another you need to normalize them). Then you could obtained normalized X by simply doing:
X = [rand(100,1) 100*rand(100,1)];
% Take a look at how different the scales of X are
figure, plot(X,'o'), title('Unnormalized features');
% Now we can subtract the mean, per column:
X = bsxfun(@minus,X,mean(X,1));
% And make standard deviation equal to 1, per column:
X = bsxfun(@rdivide,X,std(X,[],1));
% Now they can be compared!
figure, plot(X,'o'), title('Normalized Features');
Now, if you have NaNs in your data, the only difference is that instead of using 'mean' or 'std' function you should use nanmean and nanstd, which will ignore the nans in your data:
% First let's put some NaNs
X = [rand(100,1) 100*rand(100,1)];
X(round(rand(20,1)*(numel(X)-1))+1) = NaN;
% And now normalize:
X = bsxfun(@minus,X,nanmean(X,1));
X = bsxfun(@rdivide,X,nanstd(X,[],1));
figure, plot(X,'o'), title('Normalized Features');
Remember that X, even after using nanmean and nanstd STILL HAS NaNs!!! If you wanna delete them, from each column:
X_without_nans = arrayfun(@(col) X(~isnan(X(:,col)),col),(1:size(X,2)),'un',0);

Steven Lord
Steven Lord 2018-11-15
If you're using release R2018a or later, you can call normalize.
test = [NaN, NaN, 25.0182, 8.1078, 0.7304, -8.4954, ...
13.1454, 31.7136, -7.2601, 10.2586, 16.4023, -55.0183, ...
-53.4840, -4.5846, NaN];
N = normalize(test) % zscore is the default normalization method
M = mean(N, 'omitnan')
S = std(N, 'omitnan')

Pablo Rivas
Pablo Rivas 2014-7-16
编辑:Pablo Rivas 2014-7-16
Dear Kate, This is what I've done in the past:
if any(isnan(X(:)))
xmu=nanmean(X);
xsigma=nanstd(X);
x=(X-repmat(xmu,length(X),1))./repmat(xsigma,length(X),1);
else
[x,xmu,xsigma]=zscore(X);
end
First, check if there is really a NaN in your data, X, or else, just use the traditional zscore function, your zscored data is now in x. But if there are NaNs use the functions Sean suggested. I hope this helps. Note: if your data is very large, using repmat may not be the best alternative, you are better off using for loops to subtract the mean and divide by the standard deviation; it will take longer, but it is computationally feasible for large data.
Peace.

Sean de Wolski
Sean de Wolski 2013-11-11
编辑:Sean de Wolski 2013-11-11
I think any function that will calculate the zscore will inevitably have to remove the nans.
You may find nanvar, nanmean, nan* etc. useful.
  4 个评论
Sean de Wolski
Sean de Wolski 2013-11-12
I mean, this is about as simple as it gets:
x = randn(100,1);
x(rand(100,1)>0.95) = nan; %add nans
zscore(x(~isnan(x)))

请先登录,再进行评论。

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by