EXTRACTING NETCDF DATA BASED ON TIME

Question

NATHAN MURRY 2017-7-5

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347505-extracting-netcdf-data-based-on-time

评论： Mehak S 2024-4-8

Good afternoon:

I am operating on a NetCDF file that contains data for 20 variables over a period of 8 months. This is too much data, so I am trying to extract data based on the time of day. That is, to extract data for all variables from 11pm to 4am for each day in the data file. I have been able to pull out the date / time in the format "dd-mmm-yyyy hh:mm:ss". I can extract and work on ranges of time, but not a range of time per day, for many days.

I can see in my head how to do this, but I am unsure of an efficient way to code it. Experimenting with different time functions, (datenum,hours, datetime, datevec) with other structure and NetCDF tools have been unsucessful. I could use a shove in the right direction. Thank you.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2017-7-5

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347505-extracting-netcdf-data-based-on-time#answer_272996

See https://www.mathworks.com/matlabcentral/answers/312198-how-to-extract-data-from-nc-file-based-on-latitude-longitude-time-and-wind#comment_464820 and note that in my sample "expanding the selection" code that you could code hours and minutes into the from date and to date strings.

21 个评论
显示 19更早的评论隐藏 19更早的评论

NATHAN MURRY 2017-7-5

编辑：NATHAN MURRY 2017-7-5

在 MATLAB Online 中打开

Hi Walter:

It appears I spoke a bit too soon. I have been working with the 'expanded selection' code you referenced in the previous post. I added an experimental time to the 'from_' and 'to_' strings as follows.

from_date = '2015-07-01 1:0:0'; to_date = '2015-08-31 5:0:0';
time_datenum = time / (60*60*24) + datenum('1900-01-01 0:0:0')
date_match = time_datenum >= datenum(from_date) && time_datenum <= datenum(to_date);

This results in a "Operands to the and && operators must be convertible to logical scalar values" error.

The time calculations prior to the 'date_match' statement return the correct dates and times. The time variable in my NetCDF file is as follows:

 time
     Size:       10964x1
           Dimensions: obs
           Datatype:   double
           Attributes:
                       _FillValue    = -9999999
                       long_name     = 'time'
                       standard_name = 'time'
                       units         = 'seconds since 1900-01-01 0:0:0'
                       calendar      = 'gregorian'
                       axis          = 'T'

The idea makes sense, but the syntax is tripping me up. Also, I am unsure how the 'date_match' statement will grab the data for the same block of time for every day in the dataset. Thank you for your assistance.

NATHAN MURRY 2017-7-12

在 MATLAB Online 中打开

Hi Walter:

Thank you for your responses. I am back at this again, and I can now define time periods and retrieve data accordingly.

 % VARIABLES
 nctime = ncread(ncfile,'time');
 dtime = nctime/(60*60*24)+datenum(1900,1,1);
 pressure = ncread(ncfile,'ctdpf_ckl_seawater_pressure');
 temperature = ncread(ncfile,'ctdpf_ckl_seawater_temperature');
 salinity = ncread(ncfile,'ctdpf_ckl_sci_water_pracsal');
 % TIME BLOCK
 from_date = '2015-06-01 02:30:00';
 to_date = '2015-06-01 05:30:00';
 time_match = dtime >= datenum(from_date) & dtime <= datenum(to_date);
 select_dtime = dtime(time_match);
 select_pressure = pressure(time_match);
 select_temperature = temperature(time_match);
 select_salinity = salinity(time_match);

However, I am still unable to retrieve data for a particular time block for all the days contained inside a given data file. The Squeeze command makes sense, but only for particular variables, IE, salinity:

select_salinity = squeeze(salinity(time_match))

Adding (:, :, ....) as in your example generates a 'matrix exceeds dimensions', which I would expect since my select_ statements only address one variable. I am not sure how to read multiple NetCDF variables in one statement such that I can use 'Squeeze' as you did:

select_data = squeeze(all-required-variables(:, :, (however many indices), time_match, :, .....)

A solution could be to write a loop to step through all days in a data set, 'Squeezing' the data as per the time block defined above, for all required variables. However, that doesn't seem like efficient coding. I am looking for another shove in the right direction. Thanks.

NATHAN MURRY 2017-7-21

编辑：NATHAN MURRY 2017-7-21

在 MATLAB Online 中打开

Hi Walter:

I have attacked this problem two ways. I believe first is close. When using the ENTIRE data file, the script will pull the hours and data I want, exactly as I expect it. However, when I attempt to subset the date range, something goes haywire:

 %----VECTORIZE DATA FILE TIME VARIABLE---- 
 dtime_vec = datevec(dtime);  % Vectorize entire data file time variable
 %----SELECT DATE RANGE IF DESIRED----
 from_date = '2015-05-01 00:00:00';
 to_date = '2015-05-02 00:00:00';
 date_match = dtime >= datenum(from_date) & dtime <= datenum(to_date);
 date_range = dtime(date_match);
 date_range_vec = datevec(date_range);
 %----SELECT DATA BY TIME----
 from_hour = 2;
 to_hour = 4;
 %****IN 'time_match' STATEMENT BELOW, REPLACE 'date_range_vec'
 %**** WITH 'dtime_vec' IF ENTIRE DATA FILE IS TO BE USED
 time_match = date_range_vec(:,4) >= from_hour & date_range_vec(:,4) <= to_hour ;  
 time_range = datenum(datevec(dtime(time_match)));
 time_range_pressure = pressure(time_match);
 time_range_temperature = temperature(time_match);
 time_range_salinity = salinity(time_match);
 time_range_data = [time_range time_range_pressure time_range_temperature time_range_salinity];

Again, this method works perfectly when using an entire data file, without the date subsetting.

I am still working with the second method, which is adapting a stock script found elsewhere. The idea was to vectorize the full 'dtime' variable as above, and use 'find' to isolate/match the 'hour' data, and then 'ncread' to pull in the corresponding data. This works perfectly for any date range (the date match statement is rem-ed out below), but not with pulling selected hours ranges:

 %----START / END DATES & TIMES, AND MATCHING----
 dtime_vec = datevec(dtime);
 start_dt = datenum(2015,5,1,6,00,0);
 start_dt_vec = datevec(start_dt);
 end_dt = datenum(2015,5,1,6,30,0);
 end_dt_vec = datevec(end_dt);
 %----FIND DATA IN TIME RANGE----
 % tmindex = find(dtime>=start_dt & dtime<=end_dt)  %--SUBSET BY DATE--
 tmindex = find(dtime_vec(:,4) >= start_dt_vec(:,4) & dtime_vec(:,4) <= end_dt_vec(:,4))  % --SUBSET BY TIME--
 dtime = dtime(tmindex)
 %----READ VARIABLES WITHIN THE DEFINED TIME RANGE----
 pressure = ncread(ncfile,'ctdpf_ckl_seawater_pressure',tmind(1),tmind(end)-tmind(1)+1,1);
 %--------

I don't think I can use 'find' the way I am attempting to, but I am not sure if this is close as well. I could use another shove. Thank you.

NATHAN MURRY 2017-7-25

There is a copy of it in the web directory I listed above. However, I do not believe you will find anything in it critical to solving the issue at hand.

NATHAN MURRY 2017-8-1

Hi Walter: So I see I had the correct two statements, but I didn't try to join them together in a larger logical statement as you showed. With some further experimentation and additions, the function works great.

Thank you again for all of your help. I learned quite a bit in wrestling through this problem. Take care.

--NMM

请先登录，再进行评论。

Answer 2

Tanziha Mahjabin 2020-1-29

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347505-extracting-netcdf-data-based-on-time#answer_412651

编辑：Walter Roberson 2020-1-29

在 MATLAB Online 中打开

Hi,

I want to cut some time from a bid data, using ncread(source,varname,start,count).

for your information,

UCUR_sd

Size: 69x69x45588

Dimensions: J,I,TIME

Datatype: single

Attributes:

long_name = 'Standard deviation of sea water velocity U component values in 1 hour.'

units = 'm s-1'

valid_min = -10

valid_max = 10

cell_methods = 'TIME: standard_deviation'

coordinates = 'TIME LATITUDE LONGITUDE'

_FillValue = 999999

ancillary_variables = 'NOBS1 NOBS2 UCUR_quality_control'

Now if i write,

u=ncread(ncfile,'UCUR',[1 1 1],[Inf Inf 44931]);

it takes the command as the start time is from the start.

But what should i write if i want cut the time from somewhere middle?

I tried to define index,

ind=find(time>=datenum(2017,02,16,0,0,0)&time<=datenum(2017,02,17,0,0,0))
u=ncread(ncfile,'UCUR',[1 1 ind],[Inf Inf 44931]);

But it is not working. Any helpful suggestion please.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Walter Roberson 2020-1-29

netcdf times are never in MATLAB serial datenum . Instead they are in some time units relative to a particular epoch that is defined in the attributes, such as "seconds since Jul 1, 1983 00:00:00 UTC" . You need to examine the attributes for the TIME coordinate and do the conversion.

请先登录，再进行评论。

Answer 3

Tanziha Mahjabin 2020-1-30

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/347505-extracting-netcdf-data-based-on-time#answer_412783

在 MATLAB Online 中打开

Hi Walter,

Thanks for the comment. I did the conversion.

ncfile='IMOS_aggregation_20200124T074252Z.nc'; 
rtime=ncread(ncfile,'TIME');
time=datenum(rtime+datenum(1950,1,1,0,0,0));

When i write something like this, ru=ncread(ncfile,'UCUR',[1 1 1],[Inf Inf 931]); it works as the time starts from the beginning.

But i want to start the time from somewhere else as i mentioned in my question. So i defined index and try to start according to that.

ind=find(time>=datenum(2017,02,16,0,0,0)&time<=datenum(2017,02,17,0,0,0))
u=ncread(ncfile,'UCUR',[1 1 ind],[Inf Inf 44931]);

It didn't work.

8 个评论
显示 6更早的评论隐藏 6更早的评论

Walter Roberson 2020-1-30

I don't think you want that read inside a for loop??

Mehak S 2024-4-8

Why 't0-1' and not t0 while reading the file?

请先登录，再进行评论。

EXTRACTING NETCDF DATA BASED ON TIME

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

21 个评论
显示 19更早的评论隐藏 19更早的评论

更多回答（2 个）

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

8 个评论
显示 6更早的评论隐藏 6更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

EXTRACTING NETCDF DATA BASED ON TIME

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

21 个评论 显示 19更早的评论隐藏 19更早的评论

更多回答（2 个）

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

8 个评论 显示 6更早的评论隐藏 6更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

21 个评论
显示 19更早的评论隐藏 19更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

8 个评论
显示 6更早的评论隐藏 6更早的评论