I am having trouble using curl to get data, I think because of redirects and credentials.
I started with this documentation:
https://wiki.earthdata.nasa.gov/display/EL/How+To+Access+Data+With+cURL+And+Wget
Firstly I get the granules:
https://cmr.earthdata.nasa.gov/search/granules.umm_json?collection_concept_id=C2075141605-POCLOUD&temporal=2023-05-29T12:18:31Z,2023-05-30T00:18:31Z
Which gives for example:
https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc
What I can't successfully do is use curl to download this:
curl -u username:password https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc -o tmp.nc
There is a note in the curl documentation:
"When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won't be able to intercept the user+password."
I have tried with -L, --location-trusted, .netrc etc.
wget works, for example
wget --http-user=xxx --http-password=xxx https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc
Has anyone solved this issue?
Curl redirect and credential issuesforum
-
- Subject Matter Expert
- Posts: 16
- Joined: Tue Mar 14, 2023 1:41 pm America/New_York
Re: Curl redirect and credential issuesforum
Hi Nick,
As indicated on the wiki, you need to supply the cookies file written during the previous step. This command works for me:
curl -O -b ~/.urs_cookies -L -n https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc
Let us know if you have other questions, thanks
Jack
As indicated on the wiki, you need to supply the cookies file written during the previous step. This command works for me:
curl -O -b ~/.urs_cookies -L -n https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/ASCATC-L2-Coastal/ascat_20230529_180300_metopc_23651_eps_o_coa_3301_ovw.l2.nc
Let us know if you have other questions, thanks
Jack
Re: Curl redirect and credential issuesforum
Hello,
Now it works, so I don't know why it didn't earlier when I tried it.
Thanks!
Nick
Now it works, so I don't know why it didn't earlier when I tried it.
Thanks!
Nick