How to download multiple data files at once using matlab

Question

Edward McLellan 2019-10-1

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/483079-how-to-download-multiple-data-files-at-once-using-matlab

评论： Belinda Finlay 2020-4-2

I need to call on a website and download data for each and every day for the month of july. Im curious what command or format can help me do this. For example this is the code to retrieve 1 day of the year.

filename='weather_data_2019_255.txt';

if isfile(filename)

fprintf('already have the file |%s|\n',filename);

else

url='http://www.weather.unh.edu/data/2019/255.txt';

outname=websave(filename,url);

fprintf('got weather data file |%s|\n',outname);

end

How can i change the filename to allow me to download filename='weather_data_2019_(182-212).txt. I'm assuming I need to create a start date and a end date and create an output filename. May someone teach me how I can do this. Thank you very much.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

meghannmarie 2019-10-1

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/483079-how-to-download-multiple-data-files-at-once-using-matlab#answer_394345

Try this. I also put where you could change year also.

year = 2019;
start_jd = 182;
end_jd = 212;
for jd = start_jd:end_jd
    filename=['weather_data_' num2str(year) '_' num2str(jd) '.txt'];
    if isfile(filename)
        fprintf('already have the file |%s|\n',filename);
    else
        url=['http://www.weather.unh.edu/data/' num2str(year) '/' num2str(jd) '.txt'];
        outname=websave(filename,url);
        fprintf('got weather data file |%s|\n',outname);
    end
    
end

3 个评论
显示 1更早的评论隐藏 1更早的评论

Belinda Finlay 2020-3-30

在 MATLAB Online 中打开

HI Meghanmarie,

I am trying to do the same activity and your code has been an amazing help. I am getting stuck as my url has two variables.

I am assuming the error is in how I express the for two varialbes, may I request your wisdom?

year = 2018;
month = 10;
start_cd = 06;
end_cd = 15;
time = 0900;
start_jd = 279;
end_jd = 289;
for cd = start_cd:end_cd
    jd = start_jd:end_jd % my script seems to be failing here.  The example I used only had on variable and this has two, and I am not sure how to correctly format it. 
    filename = ['https://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/' num2str(year) '/' num2str(jd) '/' num2str(year) num2str(month) num2str(cd) num2str(time) '-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc']
    if isfile(filename)
        fprintf('already have the file |%s|\n',filename);
    else
        url=['https://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/' num2str(year) '/' num2str(jd) '/' num2str(year) num2str(month) num2str(cd) num2str(time) '-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'];
        outname=websave(filename,url);
         
        fprintf('got weather data file |%s|\n',outname);
    end
    
end

meghannmarie 2020-3-31

在 MATLAB Online 中打开

Is the start_cd and end_cd just the calendar day? If so, I would calculate the calender day from the julian day.

year = 2018;
month = 10;
time = 0900;
start_jd = 279;
end_jd = 289;
for jd = start_jd:end_jd
    cd = day(datetime((year-1),12,31) + days(jd));
    filename = ['https://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/' num2str(year) '/' num2str(jd) '/' num2str(year) num2str(month) num2str(cd) num2str(time) '-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc']
    if isfile(filename)
        fprintf('already have the file |%s|\n',filename);
    else
        url=['https://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/' num2str(year) '/' num2str(jd) '/' num2str(year) num2str(month) num2str(cd) num2str(time) '-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'];
        outname=websave(filename,url);
         
        fprintf('got weather data file |%s|\n',outname);
    end
    
end

Belinda Finlay 2020-4-2

在 MATLAB Online 中打开

Meghannmarie, thanks, that was a massive help.

I have all that working well; however, the data I am trying to gather is a file on a webpage, rather than a pages with data on it. When you access the webpage manually you have to click on the file to commence the download, hence when I use the code above it create a file with a html link, rather than downloading the file. Is there something you can recommend to correct this?

This the screen shot of the file I am trying to grab: (its the.nc file I am trying to download.) from herehttps://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/2018/279

my code is:

options = weboptions('Username', 'insertusername', 'Password', 'insertpassword');
year = 2018;
month = 10;
time = 0900;
start_jd = 279;
end_jd = 289;
for jd = start_jd:end_jd
    cd = day(datetime((year-1),12,31) + days(jd));
    filename = ['sst' num2str(year) '_' num2str(jd) '.nc'];
    if isfile(filename)
        fprintf('already have the file |%s|\n',filename);
    else
        url=['https://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/' num2str(year) '/' num2str(jd)];
                  outname=websave(filename,url,options);
         
        fprintf('got weather data file |%s|\n',outname);
    end
    
end

This the screen shot of the file I am trying to grab: (its the.nc file I am trying to download.) from herehttps://podaac-tools.jpl.nasa.gov/drive/files/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/2018/279

请先登录，再进行评论。