curling for MERIS/OLCI

Please enter here to ask a question about any NASA Science related topics!
Post Reply
tsmoore00
Posts: 14
Joined: Fri Apr 01, 2005 9:31 am America/New_York

curling for MERIS/OLCI

by tsmoore00 » Tue Oct 03, 2017 11:35 am America/New_York

Hi,

I am having a problem downloading MERIS or OLCI data using curl from the servers.  I can get Modis data ok, so it seems my curl is working (Mac OSX Sierra 10.12.6, curl curl 7.52.1).

I'm guessing it might be something do to with my user:pass, but I feel like I am using it ok:

curl -u username:passwd -L -O <file>

I am using my Earthdata login user and passwd info.

I get a file but its all metadata html style text that I can't decipher.

Any suggestions?

Thanks,
Tim

Tags:

gnwiii
Posts: 606
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 1

curling for MERIS/OLCI

by gnwiii » Tue Oct 03, 2017 12:37 pm America/New_York

Sounds familiar.  Registration says:

"For those users who have already created accounts via Earthdata Login, you will not need to reregister but you may be prompted to authorize a particular application/function/service on our website if you have not used it before. When that occurs, you will be redirected to the Earthdata Login page and asked to authorize that service and then once you agree, you will be automatically redirected back to our website to resume your session." 

There is  chance the html page is asking you to authorize a service.  Have you tried loading the page in a browser to see if it can decipher the important message?

tsmoore00
Posts: 14
Joined: Fri Apr 01, 2005 9:31 am America/New_York

curling for MERIS/OLCI

by tsmoore00 » Wed Oct 04, 2017 10:27 am America/New_York

Thanks.  I get redirected to the 'Agree' page, but I don't know how curl gets past that.

I can enter the first filename into a web browser, which also brings me to the 'Agree' page, which I agree with, and then a download starts.  All subsequent files entered into a browser are downloaded without redirecting to the Agree page, but curl still won't get past it.

If anyone has a way to get past the Agree page with curl, let me know!
Thanks!
Tim

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1172
Joined: Wed Sep 18, 2019 6:15 pm America/New_York

curling for MERIS/OLCI

by OB.DAAC - SeanBailey » Thu Oct 05, 2017 9:02 pm America/New_York

Tim,

See the FAQ, or look below for the short version:

Configure your username and password for authentication using a .netrc file
  $ touch ~/.netrc
  $ echo "machine urs.earthdata.nasa.gov login <uid> password <password>" > ~/.netrc
  $ chmod 0600 .netrc

where <uid> is your Earthdata Login username and <password> is your Earthdata Login password.

Call cURL with a cookie:
  $ touch ~/.urs_cookies
curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MER_RR__1PRLRA20120330_112205_000026183113_00138_52738_8486.N1.bz2 -O MER_RR__1PRLRA20120330_112205_000026183113_00138_52738_8486.N1.bz2

Sean

tsmoore00
Posts: 14
Joined: Fri Apr 01, 2005 9:31 am America/New_York

curling for MERIS/OLCI

by tsmoore00 » Fri Oct 06, 2017 8:12 am America/New_York

Great!

That seems to do the trick.

Thanks!!
Tim

Post Reply