Curl redirect and credential issuesforum

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
nick7kg
Posts: 2
Joined: Mon May 29, 2023 8:08 pm America/New_York
Answers: 0

Curl redirect and credential issuesforum

by nick7kg » Mon May 29, 2023 8:28 pm America/New_York

I am having trouble using curl to get data, I think because of redirects and credentials.

I started with this documentation:
https://wiki.earthdata.nasa.gov/display/EL/How+To+Access+Data+With+cURL+And+Wget

Firstly I get the granules:

https://cmr.earthdata.nasa.gov/search/granules.umm_json?collection_concept_id=C2075141605-POCLOUD&temporal=2023-05-29T12:18:31Z,2023-05-30T00:18:31Z

Which gives for example:
https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc

What I can't successfully do is use curl to download this:

curl -u username:password https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc -o tmp.nc

There is a note in the curl documentation:
"When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won't be able to intercept the user+password."

I have tried with -L, --location-trusted, .netrc etc.

wget works, for example

wget --http-user=xxx --http-password=xxx https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc

Has anyone solved this issue?

Tags:

PODAAC - jmcnelis
Subject Matter Expert
Subject Matter Expert
Posts: 16
Joined: Tue Mar 14, 2023 1:41 pm America/New_York
Answers: 0

Re: Curl redirect and credential issuesforum

by PODAAC - jmcnelis » Wed May 31, 2023 5:58 pm America/New_York

Hi Nick,

As indicated on the wiki, you need to supply the cookies file written during the previous step. This command works for me:

curl -O -b ~/.urs_cookies -L -n https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc

Let us know if you have other questions, thanks
Jack

nick7kg
Posts: 2
Joined: Mon May 29, 2023 8:08 pm America/New_York
Answers: 0

Re: Curl redirect and credential issuesforum

by nick7kg » Wed May 31, 2023 9:41 pm America/New_York

Hello,

Now it works, so I don't know why it didn't earlier when I tried it.

Thanks!

Nick

Post Reply