How can I Run the Code On Multiple data Files (.txt) sequentially from a folder

Question

SA 2021-3-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/782808-how-can-i-run-the-code-on-multiple-data-files-txt-sequentially-from-a-folder

编辑： SA 2021-4-2

Hi All,

I would like to write a script that will open a group (10 or 20 files at once w.r.t time sequence) of .txt files from a folder (contain some thousands of data files). So far the following script sound good but each time I need to select the 10 or 20 files manually. Is there any way to read the 10 or 20 files automatically and in this way I can read all the files inside the folder? I think there should be a smart way in matlab, Can anybody please help me to get me out from here?

Thanks in advance

[file_list, path_]=uigetfile('*.txt','Grab the file first','files Want to Select automatic','Multiselect','on');
for i=length(file_list)
    original_path =[path, file_list(i)]
end

N.B. I use Matlab 2018 version

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2021-3-26

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/782808-how-can-i-run-the-code-on-multiple-data-files-txt-sequentially-from-a-folder#answer_658596

在 MATLAB Online 中打开

per_group = 10;
p = dir('CTS_main-128361*.txt');
nFiles = numel(p);
for base_k = 1:per_group:nFiles
    group_end = min(nFiles, base_k+per_group-1);
    group_size = group_end - base_k + 1;
    A_cat = cell(group_size, 1);
    for k = base_k : group_end
      A = dlmread(p(k).name);     
      A_cat{k} = cell2mat(A);
    end
    A = vertcat(A_cat{:});
    %do something with this group of 10
end

15 个评论
显示 13更早的评论隐藏 13更早的评论

SA 2021-3-26

编辑：SA 2021-3-26

It suits for the individual file or a group of 10. These GPS time is given to the file names,

(i) is there any possible way to extract those UTC time from the files of per_group (for the huge files, how to format times)

(ii) say I plot the per_group data versus time (10*1000s= ~ 167 minutes & I put ticks on every 500 seconds). I want to see this 167 minutes to UTC time or at least put ticks@10 UTC timestamps .

format long g

T= [1283610000; 1283611000; 1283612000; ........; 1283619000]

I go forward in this way (but it performs total time stamp-ing of the the files in the directory):

myDir = uigetdir; % Get the files from directory

files = dir(fullfile(myDir,'CTS_main-*.txt'));

files = struct2table(files); % Total file info as a Table

s=files{:,1}; % 1st column of the table

timeXtract=regexp(s,'\d+','match');

timeStamp=str2double(cat(1,timeXtract{:}));

desiredTimeStamp = timeStamp(:,1);

utcTimeStamp = datetime(desiredTimeStamp, 'convertfrom', 'posixtime', 'TimeZone', 'UTC+6');

Can you help me to get me the possible way for the above group? Thanks

Walter Roberson 2021-3-27

在 MATLAB Online 中打开

Ah, this is getting obscure ;-)

A_timestamp = NaT(group_size, 1, 'TimeZone', 'UTC+6');

That should fix the "'Cannot combine or compare a datetime array with a time zone with one without a time"

The 8 leading NaT suggest that the first 8 entries do not have valid POSIX timetamps. If you try

per_group = 10;
p = dir('CTS_main-*.txt');
nFiles = numel(p);
for base_k = 1:per_group:nFiles
    group_end = min(nFiles, base_k+per_group-1);
    group_size = group_end - base_k + 1;
    A_cat = cell(group_size, 1);
    A_timestamp = NaT(group_size, 1, 'TimeZone', 'UTC+6');  %obscure!
    for k = base_k : group_end
      thisfile = p(k).name;
      timestamp = str2double(regexp(thisfile, '\d+', 'match', 'once'))
      A = dlmread(thisfile);
      A_cat{k} = A;
      A_timestamp(k) = datetime(timestamp, 'convertfrom', 'posixtime', 'TimeZone','UTC+6');
    end
    A = vertcat(A_cat{:});
    %do something with this group of 10
end

then what shows up when it displays the initial timestamps ?

I am speculating at the moment that some of the filenames might not have any digits at all.

SA 2021-3-27

编辑：SA 2021-4-2

在 MATLAB Online 中打开

In your code, if choose 'per_group=2', then for the (1st) loop A_cat= 2 1; 2 1 while for the other loop A_cat= 3 1; 4 1 (which are ordered cell). I don't understand what the reason is. Which changes made in the code to make the 1st loop output ordered like A_cat= 1 1; & 2 1, the rest cells are ordered

per_group=2;  % to fix bug in the script rather than 10, 
A_cat= 2  1
2   1; 
% while for the next loop A_cat=
3   1;
4   1  % & so on which are ordered

2. 'A_timestamp' it contains the only time format of the current loop, it can't sustain the earlier time formats. The earlier timeformats is in "NaT" format as follows:

NaT (corresponds to cell {2,1})

04-Sep-2010 16:33:20 (corresponds to cell {3,1})

04-Sep-2010 16:50:00 (corresponds to cell {4,1})

Can you please tell me the reason?

3. Another issue the timestamp here we use probably is starts calculating from 1970-01-01 midnight, but I need to set reference Jan 06, 1980.

A_timestamp(k) = datetime(timestamp, 'convertfrom', 'posixtime', 'TimeZone','UTC+6');

for this reason my time here (04-Sep-2010 16:50:00) is 10 years behind!! Do I need to convert time from "gps2utc" or you have some magical scripts?

The third issue is solved with the help of other people i.e.

A_timestamp(k) = datetime(1980,1,6,0, 0, timestamp,'TimeZone', 'UTC'); % It works well

SA 2021-3-28

编辑：SA 2021-3-30

在 MATLAB Online 中打开

I want to see my xlabel as "1 sec mean pulse from 04-Sep-2010 17:50:00 UTC (1283619000=timestamp)". Can you please help how the script look like? num2str doesn't work....for recalling 'timestamp'- it would be different. Could you please help me in this regard?

The error message comes "Error using sprintf

Unable to convert 'datetime' value to 'int64'."

Thanks in advance

title(sprintf('(%d-%d)-th Files Avg. 1s Pulse:2nd Sensor',base_k, base_k+per_group-1));
xlabel(sprintf('1s mean Pulse from UTC time #(%d to %d)',utcTimeStamp(1st of base_k), utcTimeStamp(end of base_k)); % how to insert this "A_timestamp" value in xlabel
%% How to see the xlabel as "1s mean Pulse from UTC time # (04-Sep-2010 20:20:00 to 04-Sep-2010 22:50:00)"

Walter Roberson 2021-3-30

在 MATLAB Online 中打开

xlabel(sprintf('1s mean Pulse from UTC time #(%d to %d)', posixtime(utcTimeStamp(1st of base_k)), posixtime(utcTimeStamp(end of base_k)));

请先登录，再进行评论。

Answer 2

Mohammad Sami 2021-3-25

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/782808-how-can-i-run-the-code-on-multiple-data-files-txt-sequentially-from-a-folder#answer_657613

编辑：Mohammad Sami 2021-3-26

在 MATLAB Online 中打开

From your question I assume that you want to essentially read all the files, not just 10-20 files.

You can essentially use uigetdir to get the folder that you want to load the files from.

Then use the dir function to list all the txt files inside the selected folder.

p = uigetdir; % updated var from path to p
files = dir(fullfile(p,'*.txt'));
data = cell(length(files),1);
for i = 1:length(files)
filepath = fullfile(files(i).folder,files(i).name);
data{i} = readtable(filepath); % assuming your data can be read with readtable.
end
% do something with your data.

5 个评论
显示 3更早的评论隐藏 3更早的评论

SA 2021-3-25

在 MATLAB Online 中打开

p=dir('CTS_main-128361*.txt');
nFiles=numel(p);
A_cat=cell(nFiles,1);
for k=1:nFiles
    A=dlmread(p(k).name);     
    %A=textscan(p(k).name, '%f %f')     
    A_cat{k} = cell2mat(A); disp(size(A_cat))
end
A = vertcat(A_cat{:});

with this code I can call 10 files (e.g. ...61000, ....619000........ ), but how can I call the next 10 files, and so on?

Mohammad Sami 2021-3-26

在 MATLAB Online 中打开

p = uigetdir; % changed var from path to p
% based on your comments assuming your file names are like
% CTS_main-1234567890.txt
files = dir(fullfile(p,'CTS_main-*.txt'));
files = struct2table(files);
files.id = str2double(extractBetween(files.name,'CTS_main-','.txt');
files.g = floor(files.id / 10000); % create file grouping. adjust groups here
ug = unique(g);
for j = 1:length(ug)
    % load one group of data
    k = files.g == ug(j);
    filesg = files(k,:);
    data = cell(length(filesg),1);
    for i = 1:length(filesg)
    filepath = fullfile(filesg.folder{i},filesg.name{i});
    data{i} = readtable(filepath); % assuming your data can be read with readtable.
    end
    data = vertcat(data{:});
    % do something with the current group of data
end

请先登录，再进行评论。

How can I Run the Code On Multiple data Files (.txt) sequentially from a folder

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

15 个评论
显示 13更早的评论隐藏 13更早的评论

更多回答（1 个）

5 个评论
显示 3更早的评论隐藏 3更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

How can I Run the Code On Multiple data Files (.txt) sequentially from a folder

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

15 个评论 显示 13更早的评论隐藏 13更早的评论

更多回答（1 个）

5 个评论 显示 3更早的评论隐藏 3更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

15 个评论
显示 13更早的评论隐藏 13更早的评论

5 个评论
显示 3更早的评论隐藏 3更早的评论