creating a loop for a specific computation

1 次查看（过去 30 天）
sermet OGUTCU
sermet OGUTCU 2021-7-31
评论： dpb 2021-8-2
tCOD=readtable(full_file_name,'FileType','text', ...
'headerlines',end_of_header_line,'readvariablenames',0);
all_time=tCOD{:,3:8};
all_time_second= all_time(:,4)*3600+all_time(:,5)*60+all_time(:,6); % seconds
unique_seconds=unique(all_time_second);
The attached data file can be read by the above codes and "unique_seconds" variable can be created correctly. When multiple data files are involved, I used the following codes;
for j=1:num_of_files
tCOD{j,:}=readtable(full_file_name(j,:),'FileType','text', ...
'headerlines',end_of_header_line(j),'readvariablenames',0);
end
all_time_0=vertcat(tCOD{:});
all_time=all_time_0{:,3:8};
all_time_second= all_time(:,4)*3600+all_time(:,5)*60+all_time(:,6); % seconds
For multiple files, all_time_second consists of n x 1 array where n equals the row numbers of tCOD.
for k=1:num_of_files
[a(k),b(k)]=size(tCOD{k,:});
end
where a consists of row size of each tCOD array (378063 and 377840 for two different data files). I need to apply unique(all_time_second) command for each independent n x 1 array from tCOD as follows;
for example if two files are read;
unique_seconds_1=unique(all_time_second(1:a(1)));
unique_seconds_2=unique(all_time_second(a(1)+1:a(1)+a(2)));
for example if three files are read;
unique_seconds_1=unique(all_time_second(1:a(1)));
unique_seconds_2=unique(all_time_second(a(1)+1:a(1)+a(2)));
unique_seconds_3=unique(all_time_second(a(1)+a(2)+1:a(1)+a(2)+a(3)));
How I can create a loop or something else to create unique_seconds arrays w.r.t. the file numbers without explicitly writing unique_seconds_i?
I attached the one representative data file.
  8 个评论
显示 6更早的评论
dpb
dpb 2021-7-31
OK. But... :) (always a rhetorical "but" isn't there... <VBG>)
One could do something like that, but instead let me ask what the purpose in building those vectors of unique times is? Are you wanting to merge/select data from the various files on the basis of matching them? If so, there are much simpler ways to do so that don't rely on building such arrays.
Also, you don't need the sizes() arrays and the linear indexing even to do whaty you are doing -- set membership functions such as intersect() are vectorized and would work over the indivdual file content without the combining of the data.
But, before we go down that rabbit hole with Alice, let's also make sure we can't just go to the tea party instead... :)
sermet OGUTCU
sermet OGUTCU 2021-8-1
编辑：sermet OGUTCU 2021-8-1
The purpose of creating the unique times (unique_seconds array) is just to compute how many unique times (different times) are there in the data file (it is 2880 for the single attached data). In this way, I can know the number of independent time sets in data file (there are bunch of repeated time sets in the data file). The above codes are purely intended to do so. When multiple files are read, I need to compute the number of each file's independent time sets (for example, 2880,2880,2500, etc.). If there are other more convenience way to do so, I don't necessarily use the above codes.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

dpb
dpb 2021-8-1
编辑：dpb 2021-8-2
OK, just making sure...although it may be interesting, still not positive it's of all that much use in the end...to collect data at similar times of day across days where may not be the same number of observations still will need to use join or similar. I'd think turning it into a time table and retime could be worth consideration as well...
But,anyway, just to answer the Q? asked, quit trying to figure out your purpose... :)
tCOD=[]; % an empty placeholder
datdir='YourDataStoragePath';
d=dir(fullfile(datdir,'COD_MatchWildCardString*.CLK')); % return list of wanted files
nFiles=numel(d); % how many files found
nUT=zeros(nFiles,1); % preallocate unique times number array
for j=1:nFiles
ffn=fullfile(d.folder,d.name); % get fully qualified file name
nHdr=nHeaderLines(ffn); % find number header lines to skip
tC=readtable(ffn,'FileType','text', ... % and read the file to temporary table
'numHeaderLines',nHdr, ...
'ReadVariablNnames',0);
% create useful variable names -- you can fill in rest I'm not sure of best
tC.Properties.VariableNames(3:8)={'Year','Month','Day','Hour','Min','Sec'};
tC.DateTime=datetime(tC{:,3:8}); % create the date time variable
nUT(j)=numel(unique(tC.DateTime)); % and count number timestamps this file
tCOD=[tCOD;tC]; % catenate the new file to end(*)
% whatever else want to do on individual file here...
end
% whatever else want to do on combined file here...
...
The helper function to get the number of header lines -- could be an internal function in the main m file or make it its own m-file if it might be useful elsewhere besides...or, if the number is fixed and known a priori, can dispense with; this adds generality if it is variable between files or groups of files.
If it is fixed and known to be so for a given group of files, can be called just once instead of inside the loop.
function nHdr=nHeaderLines(ffn)
% return number of lines in header of .CLK file -- looks for specific
% string "END OF HEADER" as last line of the header in the file...
% will fail if string not found...
fid=fopen(ffn); % open file to read
nHdr=1; % initialize counter
while ~contains(fgetl(fid),'END OF HEADER') % look for the end of the header string
nHdr=nHdr+1;
end
fid=fclose(fid);
end
(*) NB: The catenation of the files this way may slow things down as they're pretty big; if this is turns out to be a serious problem, post back....this isn't the most efficient but the easiest to code and if only doing once may be good enough.
When get a large file built, be sure to save it in .mat file and then won't have to rebuild it and so loading will be much quicker.
Again, might want to consider a timetable here depending upon just what are next step(s)...
  3 个评论
显示 1更早的评论
dpb
dpb 2021-8-2
No problem...I also just noticed (and corrected little bit ago) that left out the unique call in argument of numel for nUT...so it'll be the height of the table, not the count intended.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Data Type Conversion 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by