downloading files from a website with conditions on names of files

1 次查看（过去 30 天）
alpedhuez
alpedhuez 2022-2-9
评论： Walter Roberson 2022-3-12
Question: I work on a website https://www.somecompany.com/xml/.
This directory has files whose filename starts with a letter "A" and "B".
The filenames in the directory are like:
A_20080403.xml
A_20080403_1.xml
A_20080403_2.xml
A_20080404_1.xml
B_20080403_1.xml
That is
  • Filenames are of the form "Capital letters"+"_"+"date"+"_"+"numbers".xml or "Capital letters"+"_"+"date".xml
  • There are dates that do not have corresponding files
I would like to download all the files whose filenames start with a letter "A".
What has been tried:
(a) I was able to save a single file using "websave" command.
(b) I have asked the question at https://www.mathworks.com/matlabcentral/answers/457470-writing-loops-to-download-files-using-matlab-websave?s_tid=srchtitle and I got a code
for k = 20080401:20100101
filename = sprintf('A%d.xml', k);
url = ['https://www.somecompany.com/xml/' filename];
outfilename = websave(filename,url);
end
Problems with the above code: The above code does not work because
  • This code assumes the filename of the form "Capital letters"+"date".xml and not the filenames that explained above
  • This code returns the error for a date when there are no corresponding files and stops then
How shall one improve the above code?

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2022-2-9
It would be more robust / faster if the site provided a way to list the available files, instead of having to do trial and error.
baseurl = "https://www.somecompany.com/xml/";
datelimits = datetime({'20080401', '20100101'}, 'InputFormat', 'yyyyMMdd');
subfile_limit = 5; %no more than _5 -- adjust as appropriate
subfile_modifier = ["", "_" + (1:subfile_limit)] + ".xml";
for Day = datelimits(1):datelimits(2)
daystr = string(Day);
for Sub = subfile_modifier
filename = "A_" + daystr + Sub;
url = baseurl + filename;
try
outfilename = websave(filename,url);
fprintf('fetched %s\n', filename);
catch
break; %skip remaining subfiles for this date upon first failure
end
end
end
  2 个评论
alpedhuez
alpedhuez 2022-3-12
When I ran the code above, I got the value of
daystr "01-Jan-2010"
filename "A_01-Jan-2010.xml"
But the filename is, as noted above, of the format of
A_20100101.xml
Thus one needs to modify the process of assigning the variable daystr.
Walter Roberson
Walter Roberson 2022-3-12
datelimits = datetime({'20080401', '20100101'}, 'InputFormat', 'yyyyMMdd', 'Format', 'yyyyMMdd');

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Downloads 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by