wget 403 forbidden

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
katiemcquillan
Posts: 2
Joined: Thu Mar 31, 2022 9:51 am America/New_York
Answers: 0

wget 403 forbidden

by katiemcquillan » Mon Apr 04, 2022 4:07 pm America/New_York

I am trying to download files I ordered through AppEEARS using wget. I'm getting a 403 forbidden error and I can't figure out why. I've used wget for this exact procedure before and never had any issues. Here is the error message when I try to download a single file:

(base) katiemcquillan@Katies-MacBook-Pro HubbardBrook % wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --auth-no-challenge=on --keep-session-cookies --content-disposition --verbose https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)
--2022-04-04 15:43:59-- https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)
Resolving appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)... 99.84.111.29, 99.84.111.48, 99.84.111.125, ...
Connecting to appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)|99.84.111.29|:443... connected.
HTTP request sent, awaiting response... 403 FORBIDDEN
2022-04-04 15:43:59 ERROR 403: FORBIDDEN.


I wasn't sure if I made the .netrc / cookies files correctly so I tried with my username and password and got basically the same error:
(base) katiemcquillan@Katies-MacBook-Pro HubbardBrook % wget user=katiemcquillan password=fakepassword --auth-no-challenge=on --keep-session-cookies --content-disposition --verbose https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)

--2022-04-04 16:03:20-- http://user=katiemcquillan/
Resolving user=katiemcquillan (user=katiemcquillan)... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ‘user=katiemcquillan’
--2022-04-04 16:03:20-- http://password=fakepassword/
Resolving password=fakepassword (password=fakepassword)... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ‘password=fakepassword’
--2022-04-04 16:03:20-- https://appeears.earthdatacloud.nasa.gov (/api/bundle/6d9a8fc2-e993-45c4-8e5e-9b6ccb7d99ad/a8857e3b-1f99-429a-abe8-429d6fabde5e/ECO2LSTE.001_SDS_QC_doy2021001123930_aid0001.tif)
Resolving appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)... 99.84.111.29, 99.84.111.55, 99.84.111.48, ...
Connecting to appeears.earthdatacloud.nasa.gov (appeears.earthdatacloud.nasa.gov)|99.84.111.29|:443... connected.
HTTP request sent, awaiting response... 403 FORBIDDEN
2022-04-04 16:03:20 ERROR 403: FORBIDDEN.


What's weird is it seems unable to find my username when I'm positive that it's correct.

Any help troubleshooting this would be greatly appreciated!

Tags:

LP DAAC - lien
User Services
User Services
Posts: 170
Joined: Thu Jun 25, 2020 9:51 am America/New_York
Answers: 0
Been thanked: 4 times

Re: wget 403 forbidden

by LP DAAC - lien » Tue Apr 05, 2022 7:55 am America/New_York

Hi Katie,
Just want to see if you were able to try what I suggested yesterday:
Here are a couple things we see that may be causing causing issues, can you give this a try

1.You will need to submit a token, not a username and password, to download an AppEEARS output file. To get this token you can run the following with wget

> wget -q -O - --method POST --user=USERNAME --password=PASSWORD --auth-no-challenge https://appeears.earthdatacloud.nasa.gov/api/login

If successful a token will be provided, along with a token_type and expiration which are not needed. This command seems to be finicky. If there are any problems with the command above, try removing all of the spaces between arguments and re-adding them.

2. With the token you can now use wget to download the files within the text file. See below:

> wget --header "Authorization: Bearer TOKEN" -i HBInstantaneous2019-download-list.txt

Replace TOKEN in the command above with the token from the login request.

Please let me know if it still is being a problem.


Also, if you were and it did not work yesterday, could you try again today. We were experiencing database hardware issues since Friday. We had the new hardware installed last evening and those issues should be behind us.
Thanks,
Brett

katiemcquillan
Posts: 2
Joined: Thu Mar 31, 2022 9:51 am America/New_York
Answers: 0

Re: wget 403 forbidden

by katiemcquillan » Wed Apr 06, 2022 8:39 am America/New_York

This worked, thanks for your help!

jw2495
Posts: 1
Joined: Wed Apr 20, 2022 2:15 pm America/New_York
Answers: 0

Re: wget 403 forbidden

by jw2495 » Fri Jul 08, 2022 4:21 pm America/New_York

Hi Katie and Brett,

I am having exactly the same issue and feel so lucky to find this thread. However, for some reason I still cannot get the token with the command.

I tried "wget -q -O - --method POST --user=USERNAME --password=PASSWORD --auth-no-challenge https://appeears.earthdatacloud.nasa.gov", with my own username/password filled in.
I got an error message:
"wget: unrecognized option '--method'
Usage: wget [OPTION]... [URL]..."

If I remove the dash in front of --method, there is no error message but nothing shows. So I still don't get a token to run the second command (or I just don't know where to find the token?)
"wget --header "Authorization: Bearer TOKEN" -i xxxx-download-list.txt"

Could someone give me a hint to solve this problem? Your response is highly appreciated!

Thanks in advance,
Jiaming

piscini
Posts: 1
Joined: Mon May 22, 2023 8:58 am America/New_York
Answers: 0

Re: wget 403 forbidden

by piscini » Mon May 22, 2023 9:02 am America/New_York

Hello,
I experienced the same error. I have txt list file but wget fails when try do download ECOSTRESS data.

I tried the following solution:
wget -q -O - --method POST --user=USERNAME --password=PASSWORD --auth-no-challenge https://appeears.earthdatacloud.nasa.gov

using my credentials in place of USERNAME and PASSWORD, but no TOKEN is generated
Do You have any idea why?
thanks,
Alessandro

LP DAAC - dgolon
User Services
User Services
Posts: 283
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 15 times
Been thanked: 2 times
Contact:

Re: wget 403 forbidden

by LP DAAC - dgolon » Tue Jun 06, 2023 9:57 am America/New_York

Hello @piscini @jw2495 Please update the link in your initial response to https://appeears.earthdatacloud.nasa.gov/api/login That should solve the issue but let us know if you are still seeing issues. Thank you - Danielle
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

Post Reply