Trouble with Authentication Error when Accessing MERRA-2 Data via OPenDAP URL

[serena251199]

Hi,

I'm facing an authentication error when trying to retrieve weather data from Merra-2 using Python language in Jupyter Notebook on MacOS. I need a dataset with daily data and [longitude, latitude] coordinates for my weather prediction model. However, I'm having trouble with the authentication, it keeps showing this error:

"syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: HTTP^ Basic: Access denied"

Here's what I have done: I have registered for Earth Data account, specified the MERRA-2 collection and generated an OPenDAP URL to access MERRA-2 data (code snippet referenced from online resources is attached).

=> Could you please clarify on any further authentication steps I need to take to authenticate my access to the Earth Data portal? Or are there any alternative ways I can try?

Thank you so much, I appreciate it.

[GES DISC - cbattisto]

Hello,

Have you verified that your "Earthdata prerequisite files" have been generated and stored? These include the .netrc, .dodsrc, and .urs_cookies files: https://disc.gsfc.nasa.gov/information/howto?keywords=prerequisite&title=How%20to%20Generate%20Earthdata%20Prerequisite%20Files

Thank you,
Chris Battisto (NASA GES DISC)

[GES DISC - cbattisto]

Hello,

Just following up on this error. Were you able to resolve it?

Chris Battisto

[mapes]

I suspect the problem might be this unpleasant silly additional bottleneck called Authorize? https://disc.gsfc.nasa.gov/earthdata-login

[brendanwallace]

Hello, I was also running into this problem. I found I only had trouble when using xarray open_dataset and open_mfdataset. Calling netCDF4.Dataset( ) on a link opened it successfully, but I prefer to use xarray.

I followed the instructions that both cbattisto and mapes listed, but I found that I needed to do an extra step to get it to open with xarray successfully.

That step consists of:
1. from pydap.net import create_session
2. my_session = create_session()
3. xr.open_mfdataset(urls, engine='pydap', session=my_session)

I got those steps from this tutorial:
https://lb.gesdisc.eosdis.nasa.gov/meditor/notebookviewer/?notebookUrl=https://github.com/nasa/gesdisc-tutorials/blob/main/notebooks/How_to_Access_GES_DISC_Data_Using_Python.ipynb#opendap_xarray

Changing the engine to pydap finally did the trick and allowed me to access the data. Hopefully this post can be of some use to those scouring Google for a solution!

[mapes]

Wow, just to get a ds for .sel() and .plot() commands:

2 screenfuls of code, separately typing in the collection name and a version number deep into it. Also a pre-subset of space and time. (optional? why would I, why didn't the example? does performance in access to either the ds I can work with, or data access thereafter, depend on this?)

Not the xarray ideal of a workflow, but it works. Thanks for sharing that.