Download all daily data (.nc) directly from the command window

Question

Augusto Gabriel da Costa Pereira 2022-12-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1874267-download-all-daily-data-nc-directly-from-the-command-window

评论： Mathieu NOE 2022-12-13

The script below collects data for a specific latitude and longitude of precipitation for all days in January 2000 at this link: https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/ access /daily/0.25deg/2000/01/

However, my goal is not just to pull data from specific locations and turn it into .xlsx for just one month.

I want to download every day of the year directly from this link without specifying latitude and longitude and without having to do the month-by-month process: https://www.ncei.noaa.gov/data/cmorph-high-resolution -global-precipitation-estimates /access /daily /0.25 degrees/

% Download the CPC data used in the script below
year_start = 20000101; % first year to download
year_stop = 20000131; % last year to download
nb_of_years = year_stop - year_start +1;
httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/2000/01/";
filename_tmp="temp.nc"; % temporary filename (for loop); data will be always stored under this temp name
LG=[0.125];  % longitude (NB range is 0 : 360°)
LT=[-55.625];     % latitude
% export results to excel
filename_export='PRP_CMORPH.xlsx';
%% main loop
for ci = 1:nb_of_years
clear Cmorph
filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", num2str(year_start+ci-1), ".nc"); % create filename according to list above
disp(strcat(filename, "  is processed"));
dataUrl = strcat(httpsUrl, filename);
data = webread(dataUrl);
filename_out = websave(filename_tmp,dataUrl); % save downloaded data
%     ncdisp(filename_tmp,'/','min');      % commented, just to avoid filling the command window
precip=ncread(filename_tmp,'cmorph');  % Dimensions: lon,lat,time
long=ncread(filename_tmp,'lon');
lat=ncread(filename_tmp,'lat');
time=ncread(filename_tmp,'time');
for i=1:3
LGG=find(long==LG(i));
LTT=find(lat==LT(i));
if isempty(LGG)||isempty(LTT)
disp('LOCATION NOT FOUND IN THIS NetCDF FILE')
break
end
% Step 5 :Convert 3 dimensional array to column vector array for one station
Precip(:,i)=precip(LGG,LTT,:);  % Unrecognized function or variable 'precip'.
X(i)={'Precipitation(mm/day)'};% label for variable
end
latt2=[LG ; LT];
time=double(time);
AA=time/24+datenum('1900-01-01 00:00:0.0');
[yy,mm,dd,~,~,~] = datevec(AA);
date1=[yy mm dd];
DD={'Year','Month','Day'};
sheet=ci;  % NB : every year on a new sheet - same file
C={'LOCATION: Cities','','','Longitude';'DATE FROM JAN-DEZ 2021','','','Latitude'};
writecell(C,filename_export,"Sheet",sheet,"Range",'A1');
writecell(DD,filename_export,"Sheet",sheet,"Range",'A3');
writematrix(latt2,filename_export,"Sheet",sheet,"Range",'E1');
writecell(X,filename_export,"Sheet",sheet,"Range",'E3');
writematrix(date1,filename_export,"Sheet",sheet,"Range",'A4');
writematrix(Precip,filename_export,"Sheet",sheet,"Range",'E4');
end

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Mathieu NOE 2022-12-9

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1874267-download-all-daily-data-nc-directly-from-the-command-window#answer_1124202

在 MATLAB Online 中打开

hello Augusto

this code will loop over all specified monthes (it determines automatically how many days are in the current month)

remains now what you want to do will the daily data ...

% Download the CPC data used in the script below
which_year = 2000; % year to download
httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/";
% filename_tmp="temp.nc"; % temporary filename (for loop); data will be always stored under this temp name
%% main loop
mm = (1:12); % all monthes in year
% mm = [1 6]; % process only some specific monthes in year
for ci = 1:numel(mm) % loop over monthes
    
    days_per_month = datenum(which_year, mm(ci)+1, 1) - datenum(which_year, mm(ci), 1); % compute how many days this month/year
    for ck = 1:days_per_month % loop over days
        
       ymd_str = strcat(sprintf('%04d', which_year), sprintf('%02d', mm(ci)), sprintf('%02d', ck)); % year month day string
       Url = strcat(httpsUrl, sprintf('%04d', which_year), "/", sprintf('%02d', mm(ci)),"/");
       filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", ymd_str ,".nc"); % create filename according to list above
       disp(strcat(filename, "  is processed"));
        dataUrl = strcat(Url, filename);
        data = webread(dataUrl);
        filename_tmp=strcat("data_", ymd_str ,".nc"); % temporary filename (for loop); data will be always stored under this temp name
        filename_out = websave(filename_tmp,dataUrl); % save downloaded data
        
        %     ncdisp(filename_tmp,'/','min');      % commented, just to avoid filling the command window
        precip=ncread(filename_tmp,'cmorph');  % Dimensions: lon,lat,time
        long=ncread(filename_tmp,'lon');
        lat=ncread(filename_tmp,'lat');
        time=ncread(filename_tmp,'time');
    
    end
    
end

12 个评论
显示 10更早的评论隐藏 10更早的评论

Augusto Gabriel da Costa Pereira 2022-12-10

在 MATLAB Online 中打开

I tried the code below but I can't download the data between the years 2000 to 2020.

%% This script downloads PRP data from the CMORPH reanalysis product
% Download the CMORPH data used in the script below
which_year = 20000101; % year start 
which_year2= 20201231; % year end
nb_of_years= which_year2 - which_year +1;
httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/";
%% main loop
mm = (1:12); % every month of the year
% mm = [1 6]; % process only some specific monthes in year
for ci = 1:numel(mm) % loop over months 
    
    days_per_month = datenum(nb_of_years, mm(ci)+1, 1) - datenum(nb_of_years, mm(ci), 1); 
    
    for cl = 1:nb_of_years  % error here?
        filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", num2str(year_start+cl-1), ".nc")
        
    for ck = 1:days_per_month 
        
       ymd_str = strcat(sprintf('%04d', nb_of_years), sprintf('%02d', mm(ci)), sprintf('%02d', ck)); % ano,mes e dia com string
       Url = strcat(httpsUrl, sprintf('%04d', nb_of_years), "/", sprintf('%02d', mm(ci)),"/");
       filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", ymd_str ,".nc"); % crie o nome do arquivo de acordo com a lista acima
       disp(strcat(filename, "  is processed"));
        dataUrl = strcat(Url, filename);
        data = webread(dataUrl);
        filename_tmp=strcat("data_", ymd_str ,".nc");
        filename_out = websave(filename_tmp,dataUrl); 
             
        precip=ncread(filename_tmp,'cmorph');  
        long=ncread(filename_tmp,'lon');
        lat=ncread(filename_tmp,'lat');
        time=ncread(filename_tmp,'time');
    
    end
    
end

Mathieu NOE 2022-12-12

在 MATLAB Online 中打开

@Augusto Gabriel da Costa Pereira

Augusto

simple extension of my previous code to multiple years case (one more for loop)

ok, nothing fancy, I have to admit...

all the best

clc
clearvars
% Download the CPC data used in the script below
which_years = (2000:2005); % years to download
httpsUrl = "https://www.ncei.noaa.gov/data/cmorph-high-resolution-global-precipitation-estimates/access/daily/0.25deg/";
% filename_tmp="temp.nc"; % temporary filename (for loop); data will be always stored under this temp name
%% main loop
mm = (1:12); % all monthes in year
% mm = [1 6]; % process only some specific monthes in year
for cc = 1:numel(which_years) % loop over years
    
    this_year = which_years(cc);
    for ci = 1:numel(mm) % loop over monthes
        days_per_month = datenum(this_year, mm(ci)+1, 1) - datenum(this_year, mm(ci), 1); % compute how many days this month/year
        for ck = 1:days_per_month % loop over days
           ymd_str = strcat(sprintf('%04d', this_year), sprintf('%02d', mm(ci)), sprintf('%02d', ck)); % year month day string
           Url = strcat(httpsUrl, sprintf('%04d', this_year), "/", sprintf('%02d', mm(ci)),"/");
           filename = strcat("CMORPH_V1.0_ADJ_0.25deg-DLY_00Z_", ymd_str ,".nc"); % create filename according to list above
           disp(strcat(filename, "  is processed"));
           
            dataUrl = strcat(Url, filename);
            data = webread(dataUrl);
            filename_tmp=strcat("data_", ymd_str ,".nc"); % temporary filename (for loop); data will be always stored under this temp name
            filename_out = websave(filename_tmp,dataUrl); % save downloaded data
            %     ncdisp(filename_tmp,'/','min');      % commented, just to avoid filling the command window
            precip=ncread(filename_tmp,'cmorph');  % Dimensions: lon,lat,time
            long=ncread(filename_tmp,'lon');
            lat=ncread(filename_tmp,'lat');
            time=ncread(filename_tmp,'time');
        end
    end 
end