Read Data and Store Variables for Hundreds of Files...

4 次查看（过去 30 天）
Michelle De Luna
Michelle De Luna 2021-6-9
评论： Walter Roberson 2021-6-9
Hi all,
I have a folder with hundreds of files. I would like to read each file individually and store the results so that, in the end, I can average all of my results to produce a matrix I can visualize.
I tried to preallocate and fill a rather large matrix (20 x 600 x 300 x 4 x number of files in folder) with my output to then average along the last dimension, but this exceeds the preferred array limits set forth by MATLAB. So, there is insufficient space on my program and computer to store an array of this size.
In essence, I would like to end up with a 20 x 600 x 300 matrix that is the average of all of the timesteps (4) included in my hundreds of files. The problem is, I don't know how best to work around the space constraints. Any help would be greatly appreciated.
A bit more on the variable I am interested in extracting from each file: the variable ('temp") has four dimensions, representing the number of levels (20), longitude (600), latitude (300), and timesteps in a day (4). In other words, I am trying to read the temperature values at all locations over all 20 levels and all four timesteps in a day (one file == one day). I then need to average all of these results along the 4th dimension (time) to produce a final 20 x 600 x 300 average of temperature over all of my files. Here is what I have so far:
Folder = 'C:\Users\my\Desktop\Folder'
FileList = dir(fullfile(Folder, '*.nc4'))
temp = zeros(21,576,361,4)
size(temp)
for iFile = 1:numel(FileList)
iFile
filename = fullfile(FileList(iFile).folder, FileList(iFile).name)
for i = 1:20
i;
temp(i,:,:,:) = ncread(filename, 'temp', [1, 1, i, 1], [inf, inf, 1, inf]);
end
end

请先登录，再回答此问题。

采纳的回答

Walter Roberson
Walter Roberson 2021-6-9
编辑：Walter Roberson 2021-6-9
Folder = 'C:\Users\my\Desktop\Folder';
FileList = dir(fullfile(Folder, '*.nc4'));
filenames = fullfile({FileList.folder}, {FileList.name});
total = 0;
nfile = numel(filenames);
for iFile = 1:nfile
filename = filenames{iFile};
thisdata = ncread(filename, 'temp');
if iFile == 1
total = thisdata;
else
total = total + thisdata;
end
end
%total is now a per-file total. Need mean per timestamp
mean_data = sum(total,3) ./ (nfile*size(total,3));
mean_data = permute(mean_data, [3 1 2]); %user wants level first
  2 个评论
Michelle De Luna
Michelle De Luna 2021-6-9
Walter, as always, thank you for your support! Your answer helped me approach my question with a different perspective. Your answer of summing the matrices and then finding the mean was correct and to-the-point! Again, thank you.
Walter Roberson
Walter Roberson 2021-6-9
mean_data = sum(total,3) ./ (nfile*size(total,3));
should probably be 4 instead of 3 in both places.
Also if ncread is returning integer datatype then double(thisdata) or else you get integer overflow.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Data Import and Analysis 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by