Earthdata Forum

After several days of intermittent instability all LAADS DAAC MODIS downloads are now reporting 500. Using the provided wget command

wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/ ... /2021/167/" --header "Authorization: Bearer redacted" -P .

Reports

Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... 198.118.243.33
Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2021-06-19 14:57:39 ERROR 500: Internal Server Error.

I have not seen any response to my related question from earlier this week viewtopic.php?f=7&t=2389&sid=f5566a36b3 ... 1ca4a5ec53. Is there a direct contact at LAADS DAAC who would be best to contact about these ongoing issues?

Hello!
A Subject Matter expert has been notified and will answer your question shortly.
Please stand by.

I tried your wget command with my token and it worked without any issue (see below). Please try again after generating a new token from ladsweb (https://ladsweb.modaps.eosdis.nasa.gov/).

wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/ ... /2021/167/" --header "Authorization: Bearer YXNhZHVsbGFo.........TBlY2VhM2UzZjE3N2Rl" -P C:\temp
--2021-06-21 09:46:59-- https://ladsweb.modaps.eosdis.nasa.gov/ ... /2021/167/
Resolving ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)... 198.118.194.40
Connecting to ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)|198.118.194.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'C:/temp/MYD09CMA/2021/167/index.html.tmp'

MYD09CMA/2021/167/index.html. [ <=> ] 32.38K --.-KB/s in 0.005s

Last-modified header missing -- time-stamps turned off.
2021-06-21 09:47:03 (6.37 MB/s) - 'C:/temp/MYD09CMA/2021/167/index.html.tmp' saved [33156]

Removing C:/temp/MYD09CMA/2021/167/index.html.tmp since it should be rejected.

--2021-06-21 09:47:03-- https://ladsweb.modaps.eosdis.nasa.gov/ ... 034349.hdf
Reusing existing connection to ladsweb.modaps.eosdis.nasa.gov:443.
HTTP request sent, awaiting response... 200 OK
Length: 387071146 (369M) [application/x-hdf]
Saving to: 'C:/temp/MYD09CMA/2021/167/MYD09CMA.A2021167.006.2021169034349.hdf'

Generating a new token seemed to work. Is there a reason why the previous token (which was generated approx 1 week ago) suddenly began failing? We use these persistent tokens in our production system which is dependent on daily Aqua and Terra downloads, is there a new token expiration strategy which is not documented? How can we debug why using the previous token to download was failing?

There is no expiration for tokens. However, LAADS infrastructure team implemented a cross-site token compatibility last night that might have caused it. The cross-site token compatibility makes it possible for token generated on one LAADS site to work on all sites.

Can you provide a specific datetime when this system configuration change was made? There have been several reports of ongoing download issues as far back as June 1 including viewtopic.php?f=7&t=2386&sid=1fae28c0fb ... 89b997671d viewtopic.php?f=7&t=2389&sid=6f4bc9ecc2 ... fa7f2bee6a and potentially viewtopic.php?f=7&t=2396&sid=85da4953bf ... 10fc2e48e9. All of these issues seem related to some ongoing configuration changes. If there is a listing of recent system updates that can be made publicly available it might assist in diagnosing these issues. Thanks.

The latest change was made between 6/21 and 6/22, however old style app key authentication was decommissioned on June 1. Please refer to:
https://ladsweb.modaps.eosdis.nasa.gov/ ... /?id=63313