Problem to access OPeNDAP data behind https:// urls

Question

Sofya Kuzmina 2020-11-23

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/658718-problem-to-access-opendap-data-behind-https-urls

编辑： Sofya Kuzmina 2022-11-9

Hi!

I'm trying to access an OPeNDAP server with the https:// handle. I have the R2019b version of matlab. As far as I understand, matlab can't access the file directly because of security reasons(?). But it seems as though sometimes my code works and extracts the variable, sometimes not. See here

>> m=1
chl_A(m)=ncread(url, 'chlor_a', [lonnumber(m)-1,latnumber(m)-1,timeM(m)-1],[1 1 1]); %this code extracts data
>> m=2
>> chl_A(m)=ncread(url, 'chlor_a', [lonnumber(m)-1,latnumber(m)-1,timeM(m)-1],[1 1 1]); %this does not
Error using netcdflib
The NetCDF library encountered an error during execution of 'open' function - 'I/O failure (NC_EIO)'. If the data source was an  OPeNDAP URL,
see the OPeNDAP Troubleshooting section in the Users Guide.
Error in netcdf.open (line 67)
        [varargout{:}] = netcdflib ( 'open', filename, varargin{1} );
Error in internal.matlab.imagesci.nc/openToRead (line 1278)
            this.ncRootid = netcdf.open(this.Filename,'NOWRITE');
Error in internal.matlab.imagesci.nc (line 121)
                    this.openToRead();
Error in ncread (line 61)
ncObj   = internal.matlab.imagesci.nc(ncFile);
 

and basically matlab does this to the whole data. I have 158 points to extract and it only extracts about 30 of them, for the others the error comes up. Why?

This code used to work when the server was a http:// one. Is there any way to fix it or bypass it?

Thanks in advance!

2 个评论
显示无隐藏无

Rashed Mohammed 2020-11-26

Hi Sofya

Could you please provide the URL and Operating System you are using so that we can replicate the issue

Sofya Kuzmina 2020-11-26

Yes, sorry:

url = 'https://rsg.pml.ac.uk/thredds/dodsC/CCI_ALL-v4.2-DAILY';

and i'm using Windows 10.

Hopefully that'll be enough now.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Rashed Mohammed 2020-11-27

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/658718-problem-to-access-opendap-data-behind-https-urls#answer_557573

Hi Sofya

Thanks for the information. I was able to reproduce the issue and I have brought this issue to the notice of our developers. They will investigate the matter further.

In the meantime, can you try upgrading to the latest release of MATLAB and see if the issue persists as I was not able to reproduce the issue in R2020b.

6 个评论
显示 4更早的评论隐藏 4更早的评论

Barbara Bentz 2021-5-2

Here's the code and error (Matlab 2020a)

myVar = 'ppt';

%enter your terraclimate OPeNDAP URL

%find this here: http://thredds.northwestknowledge.net:8080/thredds/terraclimate_aggregated.html

myURL = ['http://thredds.northwestknowledge.net:8080/thredds/dodsC/agg_terraclimate_',myVar,'_1980_CurrentYear_GLOBE.nc'];

% enter your point locations as lat/lon pairs in the following arrays

latList=[41 44 45 41]; %list of north latitudes for each point location

lonList=[-110 -100 -101 -95]; %list of East longitudes for each point location

NUM_POINTS = length(latList);

%enter your time start and time end

year_start = 1958;

month_start=1;

year_end = 2017;

month_end=12;

%===================================================

timebounds_start =datenum(year_start,month_start,1)-datenum(1900,1,1);

timebounds_end = datenum(year_end,month_end,1)-datenum(1900,1,1);

timebounds = [timebounds_start timebounds_end];

lat=ncread(myURL,'lat');

lon=ncread(myURL,'lon');

time=ncread(myURL,'time');

[Y,M,D]=datevec(time+datenum(1900,1,1));

ftimei=find(time>=timebounds(1) & time<=timebounds(2));

time = time(ftimei);

NUM_TIME= length(time);

myData = NaN(NUM_POINTS,NUM_TIME);

myLat = NaN(NUM_POINTS,1);

myLon = NaN(NUM_POINTS,1);

for i= 1:NUM_POINTS;

[dist,flati]=min(abs(lat-latList(i)));

[dist,floni]=min(abs(lon-lonList(i)));

myLat(i)=lat(flati);

myLon(i)=lon(floni);

% to change variables, see list here: http://thredds.northwestknowledge.net:8080/thredds/terraclimate_aggregated.html

%dimensions of the data should be:

myData(i,:)=ncread(myURL,myVar,[floni(1) flati(1) ftimei(1)],[length(floni) length(flati) length(ftimei)],[1 1 1]);

end;

Error using netcdflib

The NetCDF library encountered an error during execution of 'open' function - 'Malformed or inaccessible DAP DDS

(NC_EDDS)'. If the data source was an OPeNDAP URL, see the OPeNDAP Troubleshooting section in the Users Guide.

Error in netcdf.open (line 67)

[varargout{:}] = netcdflib ( 'open', filename, varargin{1} );

Error in internal.matlab.imagesci.nc/openToRead (line 1278)

this.ncRootid = netcdf.open(this.Filename,'NOWRITE');

Error in internal.matlab.imagesci.nc (line 121)

this.openToRead();

Error in ncread (line 61)

ncObj = internal.matlab.imagesci.nc(ncFile);

BN 2022-11-9

I have same problem in R2021a. Did you find any solution?

Sofya Kuzmina 2022-11-9

编辑：Sofya Kuzmina 2022-11-9

Hello everyone. Based on my experience with different servers, this problem seems to just pop up every once in a while. I have since installed R2021b and it still occurs sometimes, albeit less often.

My only solution (@BN, thanks for the nudge, your reply was the only one that managed to get to my inbox!) would be:

a) give it some time, try again later. I know, not the most elegant solution, but if your desired data is not large, perhaps a try-catch loop can work.

b) find other ways to access your data, lol. For me the only problem has been with a specific dataset (now I think about it, it was quite a big one, while other .nc files I've accessed were smaller, so maybe that's part of the issue? idk), whereas other https:// servers worked fine. Never encountered this problem with an http:// server though, so I have no idea how one could fix that...

c) make sure your url is working! access it through your brщwser to check that it does exist on the server! @Subhrajit Rath and @Barbara Bentz's errors appear to be different to mine, so maybe that's a whole different issue.

Sorry I can't be of more help.

请先登录，再进行评论。

Problem to access OPeNDAP data behind https:// urls

2 个评论
显示无隐藏无

采纳的回答

6 个评论
显示 4更早的评论隐藏 4更早的评论

更多回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Problem to access OPeNDAP data behind https:// urls

2 个评论 显示 无隐藏 无

采纳的回答

6 个评论 显示 4更早的评论隐藏 4更早的评论

更多回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

2 个评论
显示无隐藏无

6 个评论
显示 4更早的评论隐藏 4更早的评论