504 Gateway timeout error in wget

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
harshbardhan
Posts: 14
Joined: Tue Oct 26, 2021 6:43 am America/New_York
Answers: 0

504 Gateway timeout error in wget

by harshbardhan » Wed Apr 27, 2022 12:48 am America/New_York

I am trying to download my order via wget as:

C:\wget>wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov (/archive/orders/501770875/)" --header "Authorization: Bearer aGFyc2hiYXJkaGFuOmJYTXVhR0Z5YzJoaVlYSmthR0Z1UUhKbFpHbG1abTFoYVd3dVkyOXQ6MTY1MTAzMjU3ODoyMjdlYjZiZDk1NGVjYjI1YjM4YjhjNDYyNmUzOThmNTEzYmIyMmFi" -P C:\Users\HARSHBARDHAN\Downloads\L2
--2022-04-27 10:05:14-- https://ladsweb.modaps.eosdis.nasa.gov(/archive/orders/501770875/)
Resolving ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)... 198.118.194.40
Connecting to ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)|198.118.194.40|:443... connected.
HTTP request sent, awaiting response... 504 Gateway Time-out
Retrying.

--2022-04-27 10:06:17-- (try: 2) https://ladsweb.modaps.eosdis.nasa.gov (/archive/orders/501770875/)
Reusing existing connection to ladsweb.modaps.eosdis.nasa.gov:443.
HTTP request sent, awaiting response... 504 Gateway Time-out
Retrying.

--2022-04-27 10:07:19-- (try: 3) https://ladsweb.modaps.eosdis.nasa.gov (/archive/orders/501770875/)
Reusing existing connection to ladsweb.modaps.eosdis.nasa.gov:443.
HTTP request sent, awaiting response... 504 Gateway Time-out
Retrying.

--2022-04-27 10:08:22-- (try: 4) https://ladsweb.modaps.eosdis.nasa.gov (/archive/orders/501770875/)
Reusing existing connection to ladsweb.modaps.eosdis.nasa.gov:443.
HTTP request sent, awaiting response... 504 Gateway Time-out
Retrying.

and the prompt window shows try: 5, 6, ....so on

Please help with my mistake!!
Thanks

Tags:

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: 504 Gateway timeout error in wget

by LAADS_UserServices_M » Wed May 04, 2022 12:07 pm America/New_York

It is not possible to investigate your command since the order you mentioned has expired. However, you can try wget command provided on your order page under "See wget Download Command" command. So, if you go to:
https://ladsweb.modaps.eosdis.nasa.gov (/archive/orders/501775062/) -- one of your active order.
- Log in under "Profile", top right and then click on "See wget Download Command" tab, you will see complete wget command with your token already included.
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

tom_flamelit
Posts: 4
Joined: Sun May 01, 2022 11:55 pm America/New_York
Answers: 0

Re: 504 Gateway timeout error in wget

by tom_flamelit » Sun May 08, 2022 6:28 pm America/New_York

I have also been observing 504 errors on several data products on weekends, especially Sunday afternoons.

For example, right not at 5:26 PM CT on 5/8/2022, I am receiving 504s for both website and wget calls for:

* https://nrt3.modaps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_F1_NRT

* https://nrt3.modaps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_F2_NRT

* https://nrt3.modaps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_F3_NRT

* https://nrt3.modaps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_NRT

I tested both on and outside of a VPN just to see if that was the issue. It has been ongoing since about 3PM CT, and I observed a similar issue last week and with pages that are linked (but can be found via Google cache, e.g https://webcache.googleusercontent.com/search?q=cache:pj2urdiusXsJ:https://ladsweb.modaps.eosdis.nasa.gov/learn+&cd=1&hl=en&ct=clnk&gl=us is available but the main site gives an error "This server does not understand your most recent request." (ref: 66442d43344acd7ccc4ffd8a0b459cde)).

My best guess is that the LAADS system undergoes unexpected maintenance (or, maintenance that isn't being publicized where I see it) or that the system routinely goes down and isn't being logged.
Last edited by tom_flamelit on Sun May 08, 2022 6:30 pm America/New_York, edited 1 time in total.

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: 504 Gateway timeout error in wget

by LAADS_UserServices_M » Tue May 10, 2022 1:50 pm America/New_York

Have you tried recently because I just tried the links you sent and all are working normally.
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

tom_flamelit
Posts: 4
Joined: Sun May 01, 2022 11:55 pm America/New_York
Answers: 0

Re: 504 Gateway timeout error in wget

by tom_flamelit » Tue May 10, 2022 2:00 pm America/New_York

I tried recently and they are working now.

Though, it should be pointed out this is the third weekend in a row I observed that the links weren't accessible (parent links, not just these directly).

Is there a downtime schedule that could be shared?

stefandosev
Posts: 1
Joined: Mon May 09, 2022 3:01 am America/New_York
Answers: 0

Re: 504 Gateway timeout error in wget

by stefandosev » Wed May 11, 2022 4:04 am America/New_York

I'd like to ask because i have almost the same problem.
Other than error 504, i get ""Unauthorized -- Please use reference #: 0635d761abfd3480ebf2d27ef16097ad when contacting MODAPS Support".
I registered last week and still haven't been able to downloads hdfs.
I'm using latest firefox and with chrome and edge i get the same result.

harshbardhan
Posts: 14
Joined: Tue Oct 26, 2021 6:43 am America/New_York
Answers: 0

Re: 504 Gateway timeout error in wget

by harshbardhan » Wed May 11, 2022 5:52 am America/New_York

Yes, now its working perfect.
Thanks

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: 504 Gateway timeout error in wget

by LAADS_UserServices_M » Tue May 17, 2022 12:00 pm America/New_York

stefandosev wrote: Wed May 11, 2022 4:04 am America/New_York I'd like to ask because i have almost the same problem.
Other than error 504, i get ""Unauthorized -- Please use reference #: 0635d761abfd3480ebf2d27ef16097ad when contacting MODAPS Support".
I registered last week and still haven't been able to downloads hdfs.
I'm using latest firefox and with chrome and edge i get the same result.
Sorry we missed your question, do you still have issue with download data from LAADS website? You may want to consult tutorials provided on the website at:
https://ladsweb.modaps.eosdis.nasa.gov/learn
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

ervinjason
Posts: 2
Joined: Fri May 20, 2022 3:18 pm America/New_York
Answers: 0

Re: 504 Gateway timeout error in wget

by ervinjason » Fri May 20, 2022 3:26 pm America/New_York

I have also been observing 504 errors on several data products on weekends, especially Sunday afternoons.

I am receiving 504s for both website and wget calls for:

* https://nrt3.aps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_F1_NRT

I tested both on and outside of a VPN just to see if that was the issue.

My best guess is that the LAADS system undergoes unexpected maintenance (or, maintenance that isn't being publicized where I see it) or that the system routinely goes down and isn't being logged.

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: 504 Gateway timeout error in wget

by LAADS_UserServices_M » Mon May 23, 2022 7:35 am America/New_York

I just tried and both nrt3 and nrt4 servers are working normally. However, the URL you are using is not correct. Try using:
https://nrt3.modaps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_F1_NRT
or backup system:
https://nrt4.modaps.eosdis.nasa.gov/archive/allData/61/MCDWD_L3_F1_NRT
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

Locked