Create FOR loop to process and create datasets from all files found
显示 更早的评论
Forgetting the need to append all data (which will be my ulimate goal), I need to process all files found to create x number of individual datasets. How do I run the following script with a FOR loop to create these datasets??
% sets base directory and lists all files stored within its subfolders
base_dir = 'Farm34Maize/';
[status, list] = system('dir /B /S TOA5_2436.flux*.dat');
result = textscan(list, '%s', 'delimiter', '\n');
filelist = result{1};
%%Read files
% Run FOR loop to read all .dat files
for i = 1:numel(filelist);
fid = fopen(filelist{i}, 'r');
%%Extract column headers
row = fgetl(fid{i}); % Skip header row
row = fgetl(fid{i}); % Extract second row that represents columns headers
cols = textscan(row, '%q', 'Delimiter', ','); % Identify column headers
cols = strtrim(cols{1}).';
%%Extract data and timestamp
row = fgetl(fid{i}); % Skip third row
row = fgetl(fid{i}); % Skip fourth row
c = length(cols);
format = ['%q', repmat('%f', 1, c-1)];
data = textscan(fid, format, 'Delimiter', ',', 'TreatAsEmpty', {'"NAN"', '"INF"'}, 'CollectOutput', 1, 'headerlines', 1);
%%Split data and timestamp
data = [datenum(data{1}) data{2}]; % In order to work correctly, input file date formats must be 'yyyy-mm-dd HH:MM:SS' or 'yyyy/mm/dd HH:MM:SS PM' or 'y
fclose(fid{i});
end
Thanks in advance.
2 个评论
Wieger Duursema
2011-6-9
Hi Bugguts99, I am wondering how you did manage to save the data within the FOR LOOP so that they do not overwrite. So, how did you do that? Thanks, Wieger
bugguts99
2012-6-8
采纳的回答
Walter Roberson
2011-5-11
Don't use system() to get the file names, use dir()
fileinfo = dir('TOA5_2436.flux*.dat');
for K = 1 : numel(fileinfo);
thisfile = fileinfo(K).name;
fid = fopen(thisfile,'rt');
% Extract column headers
row = fgetl(fid); % Skip header row
row = fgetl(fid); % Extract second row that represents columns headers
%and so on
fclose(fid)
end
Your code was trying to index fid, which was not a cell array in your code. That would have crashed your code. The change to dir() is just better programming.
3 个评论
bugguts99
2011-5-11
bugguts99
2011-5-11
Walter Roberson
2011-5-11
Your code never uses base_dir . The system() call is *not* going to make use of that variable.
%find what is in the base directory
basedirinfo = dir(base_dir);
%throw away the info on everything that is not a directory
basedirinfo = basedirinfo([basedirinfo.isdir]);
%throw away directories "." and ".."
basedirinfo = basedirinfo(~ismember({basedirinfo.name},{'.','..'}));
%everything left is a subdirectory. Loop over all of them
data = {}; %to hold the data
for didx = 1 : length(basedirinfo)
thisdir = basedirinfo(didx).name;
subspec = sprintf('%s\%s', thisdir, 'TOA5_2436.flux*.dat');
fileinfo = dir(subspec);
numfiles = length(fileinfo);
%extend the data cell array to hold more entries
oldlen = size(data,1);
data(oldlen+numfiles) = {};
%now process the files
for fidx = 1 : numfiles
thisfile = sprintf('%s\%s', thisdir, fileinfo(fidx).name);
fid = fopen(thisfile,'rt');
% extract column headers
row = fgetl(fid); %Skip header row
..... %copy appropriate code here
thisdata = textscan(fid, format, 'Delimiter', ',', 'TreatAsEmpty', {'"NAN"', '"INF"'}, 'CollectOutput', 1, 'headerlines', 1);
thisdata = [datenum(thisdata{1}) thisdata{2}]; % In order to work correctly, input file date formats must be 'yyyy-mm-dd HH:MM:SS' or 'yyyy/mm/dd HH:MM:SS PM' or 'y
data{oldlen+fidx} = thisdata;
fclose(fid);
end %of file loop within directory
end %of directory loop
%now data{1}, data{2}, and so on are the accumulated data sets
更多回答（1 个）
Jan
2011-5-11
To learn more about the DIR command:
doc dir
There you find an example of how to scan a specified folder:
dir(fullfile(matlabroot, 'toolbox/matlab/audio/*.m')
Although this does not work recursively, it is much safer.
类别
在 帮助中心 和 File Exchange 中查找有关 File Operations 的更多信息
产品
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
