How do query timestamps when reading fileDatastore

2 次查看（过去 30 天）
Dharmesh Joshi
Dharmesh Joshi 2024-1-24
回答： Walter Roberson 2024-1-24
Hello,
I am saving a substantial amount of RAW sensor data at specific intervals into individual files. Then, I am using `fileDatastore` to read all those files and compile them into one variable, as demonstrated below. The issue I'm facing is that I have data spanning over a year, but I only want to read the last three months of data. How can I modify my code so that once it starts reading a file, if it encounters timestamps older than three months, it stops progressing through that datastore?
The purpose of this is to speed up my data compilation by avoiding unnecessary data reads.
fds = fileDatastore("Raw Sensor Data\"+deviceid,"ReadFcn",@load,"FileExtensions",".mat")
readlist=readall(fds)
readlist_size=size(readlist);
temp_live_data_downloaded_data=[];
temp_ref_data_timestamp=[];
temp_ref_data_sensordata=[];
for packet = 1:readlist_size(1)
temp_live_data_downloaded_data =[temp_live_data_downloaded_data readlist{packet}.live_data_downloaded_data];
temp_ref_data_timestamp =[temp_ref_data_timestamp readlist{packet}.ref_data_timestamp];
temp_ref_data_sensordata =[temp_ref_data_sensordata readlist{packet}.ref_data_sensordata];
end
  3 个评论
显示 1更早的评论
Dharmesh Joshi
Dharmesh Joshi 2024-1-24
When ever i download new sensor data, it is saved in a new .mat file. So each sensor will have its own folder . This folder has many files with in it. But each sensor will have its own folder.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2024-1-24
fds = fileDatastore("Raw Sensor Data\"+deviceid,"ReadFcn",@load,"FileExtensions",".mat")
Instead of passing in the directory to fileDatastore(), you would pass in a cell array of validated file names.
rdpath = "Raw Sensor Data";
dinfo = dir(fullfile(rdpath, deviceid, "*.mat"));
filenames_only = {dinfo.name};
%ADJUST THIS SECTION
%for the sake of arguement assume the first 8 characters of the file name
%are constant and the timestamp portion ends with "_"
timestamps = regexp(filenames_only, '^(?<=.{8})[^_]*', 'once');
ts_datetime = datetime(timestamps);
now = datetime('now');
now_minus_3 = now - calmonths(3);
mask = isbetween(timestamps, now_minus_3, now);
dinfo = dinfo(mask);
selected_fullnames = fullfile({dinfo.folder}, {dinfo.name});
fds = fileDatastore(selected_fullnames, "ReadFcn", @local);
The portion marked ADJUST THIS SECTION needs to be adjusted to suit your purposes. The code as-is assumes that the first 8 characters of the file name are to be ignored, and that then there is a timestamp that ends at the first "_", and that timestamp can be fed into datetime() directly. This is example code that needs to be changed according to the actual filename structure you have.

类别

在 Help Center 和 File Exchange 中查找有关 Data Import and Analysis 的更多信息

标签

产品


版本

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by