How would I manipulate the following functions to use a tall array?

4 次查看（过去 30 天）
EL
EL 2020-3-2
回答： Josh Meyer 2020-4-3
Hello,
I'm having difficulty figuring out how to turn these functions to utilize tall arrays. The only examples online about using tall arrays I've found so far have been using NYC taxi data as examples, which doesn't help me at all. I can't find examples that go beyond multiplying two columns togethor, or getting the mean of a column. I'm working with a single column, no header array of data. I essentially need to apply of series of transformations on the data.
Any help is appreciated, thanks.
% Say both files are 6,000,000,000 (6e9) data points long, representing a wave.
% I need to flatten with a linear fit every 400,000, then gather variance every 100,000
Dir = 'C:/';
Data{1} = 'File1';
Data{2} = 'File2';
for filestoread = 1:length(Data);
PathFile = fullfile(Dir, Data{filestoread};
fidi = fopen(PathFile);
while ~feof(fidi)
yraw = fscanf(fidi, '%f\n', 1200000);
yflat = detrend(reshape(yraw, 400000, []));
yflat = yflat(:);
variance = var(reshape(yflat, 100000, []));
dlmwrite(varASCII, transpose(variance), 'delimiter', '\t', '-append', 'precision', 16);
end
end

请先登录，再回答此问题。

回答（1 个）

Josh Meyer
Josh Meyer 2020-4-3
First you need to make a datastore that references the underlying files for each array (one datastore per array). Then you can convert the datastores into tall arrays with T = tall(ds).
While the movvar function (moving variance) supports tall arrays, the detrend function does not (the bottom of each doc page has an Extended Capabilities section that says whether it supports tall arrays). So to detrend the data, you would probably have to use the package function matlab.tall.movingWindow to operate on the tall array with a window function you write that does the detrending.

类别

在 Help Center 和 File Exchange 中查找有关 Tall Arrays 的更多信息

标签

产品


版本

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by