Problem with MATLAB and Earthdata OPeNDAP

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
benedettobarone
Posts: 4
Joined: Mon May 08, 2023 6:09 pm America/New_York
Answers: 0

Problem with MATLAB and Earthdata OPeNDAP

by benedettobarone » Thu May 11, 2023 8:16 pm America/New_York

I am trying to access the Earthdata OPeNDAP repository for a sea surface temperature product (OSPO Global Nighttime Foundation Sea Surface Temperature) that I previously downloaded from the PODACC OPeNDAP. While I did not have any issue with PODACC, I cannot access the Earthdata repository. For example, I tried to use the ncinfo comand to access the following granule:

https://opendap.earthdata.nasa.gov/collections/C2036877745-POCLOUD/granules/20230510000000-OSPO-L4_GHRSST-SSTfnd-Geo_Polar_Blended_Night-GLOB-v02.0-fv01.0

And I obtained the following error:
"The NetCDF library encountered an error during execution of 'open' function - 'Access failure (-77)'."

I have the Earthdata login in my .netrc and last year I used it to download ocean currents from the OSCAR data product. Today, I could access those data after one attempt, but when trying again I obtained the following error:
"The NetCDF library encountered an error during execution of 'open' function - 'DAP server error (NC_EDAPSVC)'"

Lastly, I tried the same approach with the two servers reported in the MATLAB tutorial (https://opendap.github.io/documentation/tutorials/matlab_tutorial.html) and I could not access those data either. This time, I obtained the following error:
"The NetCDF library encountered an error during execution of 'open' function - 'Not a valid data type or _FillValue type mismatch (NC_EBADTYPE)'"

I do not have problems using MATLAB to access other OPeNDAP servers (the Copernicus ones, for example). I am wondering if anyone knows the cause of this problem and/or if anyone is having similar issues using MATLAB and the EarthData OPeNDAP. Any help is appreciated.

PS: I am working on a MAC with OS Ventura and MATLAB R2021a

Tags:

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 372
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

Re: Problem with MATLAB and Earthdata OPeNDAP

by OB ODPS - towens » Tue May 16, 2023 9:52 am America/New_York

Try accessing the OPeNDAP URL with a fresh web browser window. See if there is a user authentication step that Matlab is not handling.

Tommy

benedettobarone
Posts: 4
Joined: Mon May 08, 2023 6:09 pm America/New_York
Answers: 0

Re: Problem with MATLAB and Earthdata OPeNDAP

by benedettobarone » Tue May 16, 2023 5:04 pm America/New_York

Thanks for your help, Tommy,
If I restart the browser and connect to the OPeNDAP URL, it asks for my Earthdata login. Is the browser also supposed to obtain this information from the .netrc?
After the login, I have normal access to the URL.

Benedetto

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 372
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

Re: Problem with MATLAB and Earthdata OPeNDAP

by OB ODPS - towens » Wed May 17, 2023 10:53 am America/New_York

No - the browser usually has its own cache of saved passwords. But this was just a way to demonstrate the need for authentication. I am not a Matlab user, but hopefully someone else can walk you through the steps for obtaining and using an access token in Matlab.

Tommy

PODAAC - colleen
User Services
User Services
Posts: 82
Joined: Thu May 27, 2021 2:52 pm America/New_York
Answers: 0

Re: Problem with MATLAB and Earthdata OPeNDAP

by PODAAC - colleen » Wed May 17, 2023 4:32 pm America/New_York

We are sorry to hear that you are experiencing problems while trying to access the GHRSST data.

Our SME tried the OPeNDAP URL referenced today: https://opendap.earthdata.nasa.gov/collections/C2036877745-POCLOUD/granules/20230510000000-OSPO-L4_GHRSST-SSTfnd-Geo_Polar_Blended_Night-GLOB-v02.0-fv01.0

She accessed it using her browser and wget, and was able to successfully do a direct download in nc4. However, she had to refresh her requests twice because it was encountering a server issue. The Hyrax team is aware of BES connection issue (it’s an issue with their production env), but we are unsure what their timeline is to fix it.

PO.DAAC User Services

dwhitt25
Posts: 1
Joined: Fri Jun 23, 2023 6:20 pm America/New_York
Answers: 0

Re: Problem with MATLAB and Earthdata OPeNDAP

by dwhitt25 » Fri Jun 23, 2023 6:33 pm America/New_York

Today, I unsuccessfully tried several different ways to access podaac data via opendap using Matlab 2021a on MacOS Monterey, including https://opendap.earthdata.nasa.gov/collections/C2270392799-POCLOUD/granules/ssh_grids_v2205_1993020712.dap.nc4, and the data originally referenced in the initial post of this thread.

In both cases, I am able to download the data from the opendap url using safari, chrome, and curl in the command line, similar to the last post above.

But, I cannot get access in Matlab. Like the original poster, I can access opendap files using Matlab, e.g. ncdisp('http://test.opendap.org/dap/data/nc/sst.mnmean.nc.gz'), but not those in earthdata.nasa.gov.

I also cannot get the dap4:// protocol to work in the tutorial linked in the original post.

I spent a lot of time trying to get a .dodsrc file right, but I'm not certain that it is right, and I don't think I have another netcdf/C library program to test on this machine.

Do any Matlab users know a tutorial that works through the authentication/access issues for Matlab/Opendap?

PODAAC - wenhaoli
Subject Matter Expert
Subject Matter Expert
Posts: 31
Joined: Tue May 11, 2021 12:58 pm America/New_York
Answers: 0

Re: Problem with MATLAB and Earthdata OPeNDAP

by PODAAC - wenhaoli » Tue Jun 27, 2023 5:37 pm America/New_York

I have tried the wget command on linux machine. It works for both nc files, see commands below.
% wget --http-user=username --ask-password --keep-session-cookies https://opendap.earthdata.nasa.gov/collections/C2270392799-POCLOUD/granules/ssh_grids_v2205_1993020712.dap.nc4
% wget --http-user=username --ask-password --keep-session-cookies https://opendap.earthdata.nasa.gov/collections/C2036877745-POCLOUD/granules/20230510000000-OSPO-L4_GHRSST-SSTfnd-Geo_Polar_Blended_Night-GLOB-v02.0-fv01.0

Regarding the Matlab program, you could try to use the webread() function. It will return a long list of uint8. You probably need to convert it to a structure.

Post Reply