Page 1 of 1

wget 403 forbidden

Posted: Mon Apr 04, 2022 4:07 pm America/New_York
by katiemcquillan
I am trying to download files I ordered through AppEEARS using wget. I'm getting a 403 forbidden error and I can't figure out why. I've used wget for this exact procedure before and never had any issues. Here is the error message when I try to download a single file:

(base) katiemcquillan@Katies-MacBook-Pro HubbardBrook % wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --auth-no-challenge=on --keep-session-cookies --content-disposition --verbose https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)
--2022-04-04 15:43:59-- https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)
Resolving appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)... 99.84.111.29, 99.84.111.48, 99.84.111.125, ...
Connecting to appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)|99.84.111.29|:443... connected.
HTTP request sent, awaiting response... 403 FORBIDDEN
2022-04-04 15:43:59 ERROR 403: FORBIDDEN.


I wasn't sure if I made the .netrc / cookies files correctly so I tried with my username and password and got basically the same error:
(base) katiemcquillan@Katies-MacBook-Pro HubbardBrook % wget user=katiemcquillan password=fakepassword --auth-no-challenge=on --keep-session-cookies --content-disposition --verbose https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)

--2022-04-04 16:03:20-- http://user=katiemcquillan/
Resolving user=katiemcquillan (user=katiemcquillan)... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ‘user=katiemcquillan’
--2022-04-04 16:03:20-- http://password=fakepassword/
Resolving password=fakepassword (password=fakepassword)... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ‘password=fakepassword’
--2022-04-04 16:03:20-- https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)
Resolving appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)... 99.84.111.29, 99.84.111.55, 99.84.111.48, ...
Connecting to appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)|99.84.111.29|:443... connected.
HTTP request sent, awaiting response... 403 FORBIDDEN
2022-04-04 16:03:20 ERROR 403: FORBIDDEN.


What's weird is it seems unable to find my username when I'm positive that it's correct.

Any help troubleshooting this would be greatly appreciated!

Re: wget 403 forbidden

Posted: Tue Apr 05, 2022 7:55 am America/New_York
by LP DAAC - lien
Hi Katie,
Just want to see if you were able to try what I suggested yesterday:
Here are a couple things we see that may be causing causing issues, can you give this a try

1.You will need to submit a token, not a username and password, to download an AppEEARS output file. To get this token you can run the following with wget

> wget -q -O - --method POST --user=USERNAME --password=PASSWORD --auth-no-challenge https://appeears.earthdatacloud.nasa.gov/api/login

If successful a token will be provided, along with a token_type and expiration which are not needed. This command seems to be finicky. If there are any problems with the command above, try removing all of the spaces between arguments and re-adding them.

2. With the token you can now use wget to download the files within the text file. See below:

> wget --header "Authorization: Bearer TOKEN" -i HBInstantaneous2019-download-list.txt

Replace TOKEN in the command above with the token from the login request.

Please let me know if it still is being a problem.


Also, if you were and it did not work yesterday, could you try again today. We were experiencing database hardware issues since Friday. We had the new hardware installed last evening and those issues should be behind us.
Thanks,
Brett

Re: wget 403 forbidden

Posted: Wed Apr 06, 2022 8:39 am America/New_York
by katiemcquillan
This worked, thanks for your help!

Re: wget 403 forbidden

Posted: Fri Jul 08, 2022 4:21 pm America/New_York
by jw2495
Hi Katie and Brett,

I am having exactly the same issue and feel so lucky to find this thread. However, for some reason I still cannot get the token with the command.

I tried "wget -q -O - --method POST --user=USERNAME --password=PASSWORD --auth-no-challenge https://appeears.earthdatacloud.nasa.gov", with my own username/password filled in.
I got an error message:
"wget: unrecognized option '--method'
Usage: wget [OPTION]... [URL]..."

If I remove the dash in front of --method, there is no error message but nothing shows. So I still don't get a token to run the second command (or I just don't know where to find the token?)
"wget --header "Authorization: Bearer TOKEN" -i xxxx-download-list.txt"

Could someone give me a hint to solve this problem? Your response is highly appreciated!

Thanks in advance,
Jiaming

Re: wget 403 forbidden

Posted: Mon May 22, 2023 9:02 am America/New_York
by piscini
Hello,
I experienced the same error. I have txt list file but wget fails when try do download ECOSTRESS data.

I tried the following solution:
wget -q -O - --method POST --user=USERNAME --password=PASSWORD --auth-no-challenge https://appeears.earthdatacloud.nasa.gov

using my credentials in place of USERNAME and PASSWORD, but no TOKEN is generated
Do You have any idea why?
thanks,
Alessandro

Re: wget 403 forbidden

Posted: Tue Jun 06, 2023 9:57 am America/New_York
by LP DAAC - dgolon
Hello @piscini @jw2495 Please update the link in your initial response to https://appeears.earthdatacloud.nasa.gov/api/login That should solve the issue but let us know if you are still seeing issues. Thank you - Danielle