Automatic Download from a Url when the file of the date is inputted by the user

4 次查看（过去 30 天）
Tianchu Lu
Tianchu Lu 2022-8-10
编辑： Débora Rodrigues 2022-9-20
I have written a code to download a specific data from cosmic satellite website, the code is as follows
url_https='https://data.cosmic.ucar.edu';
dataUrl=strcat(url_https,"/gnss-ro/cosmic2/nrt/level1b/2022/004/podTc2_nrt_2022_004.tar.gz");
dataFile="podTc2_nrt_2022_005.tar.gz";
FileFullPath=websave(dataFile,dataUrl);
Files=untar("F:\podtc2_data\podTc2_nrt_2022_005.tar.gz",'podTc2_nrt_2022_005');
Now this code only works for one day of the data that is provided by the data portal. is there a way such that I could create a loop that modifies a string of characters corresponent to the desired date that I am looking to work with, then the web function can read the string automatically created by MATLAB.
I was wondering if a web crawler or something similar has been done on matlab?
Overall, I have currently created a code that download a specific date of data, I am looking to code a set of loops where if I enter a specific date the code would automatically crawl the web and download the specific file that I am looking for.
I have been stuck on this section for a quite a long time, hopefully you guys could give me some advice and ideas on this matter.
  2 个评论
Jan
Jan 2022-8-10
Which part of the file name do you want to modify and how? The more details you mention, the smaller is the chance of a wrong guess.
Tianchu Lu
Tianchu Lu 2022-8-10
Thank you very much for your answer benearth, so the part that should be modified would be the part of _005, as 005 means 5th of Jan 2022 and hence 001 is 1st of Jan and so on and so forth. So the pusedocode that I had in mind was that the user( myself) would enter a specific date that the I would like to investigate into, and then a loop where matlab would urlread (https://data.cosmic.ucar.edu/gnss-ro/cosmic2/nrt/level1b/2022/) and identify the date that the user want and would then download the specific zipped file of that date. so a loop where it would urlread and identify the date inputted and download the file correspond to the inputted information. But I will defo have a play with the code that you submitted below and let you know how I got on.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Jan
Jan 2022-8-10
This can be implemented with sprintf() or compose() easily.
url_https='https://data.cosmic.ucar.edu';
for year = 2020:2022 % ??? Bold guess
for day = 1:365 % ??? Bold guess
dataUrl = sprintf('%s/gnss-ro/cosmic2/nrt/level1b/%d/%03d/podTc2_nrt_%d_%03d.tar.gz', ...
url_https, year, day, year, day);
dataFile = sprintf('podTc2_nrt_%d_%03d.tar.gz', year, day);
FileFullPath = websave(dataFile, dataUrl);
end
end
  3 个评论
显示 1更早的评论
Débora Rodrigues
Débora Rodrigues 2022-9-19
Hello!
Sorry to bother, but i have a similar problem. I am trying to download a .grib preciptation data, from this website Precipitation. I saw this discussion and tried to follow.
However i have an error:
Error using websave (line 95)
The '' protocol specified in URL, 'MERGE_CPTEC_20100101.grib2', is not supported. Specify the URL with the protocol 'http://' or 'https://'.
Error in download_precipitacao (line 19)
data = websave(data_url,data_file);
my code is:
url = 'http://ftp1.cptec.inpe.br/modelos/tempo/MERGE/GPM/DAILY';
for ano= 2010:2021
for mes= 1:12
for dia=1:30
data_url = sprintf('%s/%3d/%02d/MERGE_CPTEC_%d%02d%02d.grib2', ...
url, ano, mes, ano, mes, dia);
data_file= sprintf('MERGE_CPTEC_%d%02d%02d.grib2', ano, mes, dia);
data = websave(data_url,data_file);
end
end
end
if someone can help me, i would be very grateful, thanks!
Débora Rodrigues
Débora Rodrigues 2022-9-20
编辑：Débora Rodrigues 2022-9-20
Hello! So i solved my problem!
I changed the function websave for urlwrite, and it worked! I am living it in here in case someone with similar problens appear haha.
url = 'http://ftp1.cptec.inpe.br/modelos/tempo/MERGE/GPM/DAILY';
for ano= 2010:2021
for mes= 1:12
for dia=1:30
data_url = sprintf('%s/%3d/%02d/MERGE_CPTEC_%d%02d%02d.grib2', ...
url, ano, mes, ano, mes, dia);
data_file= sprintf('MERGE_CPTEC_%d%02d%02d.grib2', ano, mes, dia);
%data = websave(data_url,data_file);
a= urlwrite(data_url,data_file);
end
end
end

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Downloads 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by