LAADS DAAC MODIS downloads reporting 500.

[sharkins]

After several days of intermittent instability all LAADS DAAC MODIS downloads are now reporting 500. Using the provided wget command

wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/ ... /2021/167/" --header "Authorization: Bearer redacted" -P .

Reports

Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... 198.118.243.33
Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2021-06-19 14:57:39 ERROR 500: Internal Server Error.

I have not seen any response to my related question from earlier this week viewtopic.php?f=7&t=2389&sid=f5566a36b3 ... 1ca4a5ec53. Is there a direct contact at LAADS DAAC who would be best to contact about these ongoing issues?

[ASDC - ingridgs]

Hello!
A Subject Matter expert has been notified and will answer your question shortly.
Please stand by.

[LAADSx_UserServices_M]

I tried your wget command with my token and it worked without any issue (see below). Please try again after generating a new token from ladsweb (https://ladsweb.modaps.eosdis.nasa.gov/).

wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/ ... /2021/167/" --header "Authorization: Bearer YXNhZHVsbGFo.........TBlY2VhM2UzZjE3N2Rl" -P C:\temp
--2021-06-21 09:46:59-- https://ladsweb.modaps.eosdis.nasa.gov/ ... /2021/167/
Resolving ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)... 198.118.194.40
Connecting to ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)|198.118.194.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: 'C:/temp/MYD09CMA/2021/167/index.html.tmp'

MYD09CMA/2021/167/index.html. [ <=> ] 32.38K --.-KB/s in 0.005s

Last-modified header missing -- time-stamps turned off.
2021-06-21 09:47:03 (6.37 MB/s) - 'C:/temp/MYD09CMA/2021/167/index.html.tmp' saved [33156]

Removing C:/temp/MYD09CMA/2021/167/index.html.tmp since it should be rejected.

--2021-06-21 09:47:03-- https://ladsweb.modaps.eosdis.nasa.gov/ ... 034349.hdf
Reusing existing connection to ladsweb.modaps.eosdis.nasa.gov:443.
HTTP request sent, awaiting response... 200 OK
Length: 387071146 (369M) [application/x-hdf]
Saving to: 'C:/temp/MYD09CMA/2021/167/MYD09CMA.A2021167.006.2021169034349.hdf'

[sharkins]

Generating a new token seemed to work. Is there a reason why the previous token (which was generated approx 1 week ago) suddenly began failing? We use these persistent tokens in our production system which is dependent on daily Aqua and Terra downloads, is there a new token expiration strategy which is not documented? How can we debug why using the previous token to download was failing?

[LAADSx_UserServices_M]

There is no expiration for tokens. However, LAADS infrastructure team implemented a cross-site token compatibility last night that might have caused it. The cross-site token compatibility makes it possible for token generated on one LAADS site to work on all sites.

[sharkins]

Can you provide a specific datetime when this system configuration change was made? There have been several reports of ongoing download issues as far back as June 1 including viewtopic.php?f=7&t=2386&sid=1fae28c0fb ... 89b997671d viewtopic.php?f=7&t=2389&sid=6f4bc9ecc2 ... fa7f2bee6a and potentially viewtopic.php?f=7&t=2396&sid=85da4953bf ... 10fc2e48e9. All of these issues seem related to some ongoing configuration changes. If there is a listing of recent system updates that can be made publicly available it might assist in diagnosing these issues. Thanks.

[LAADSx_UserServices_M]

The latest change was made between 6/21 and 6/22, however old style app key authentication was decommissioned on June 1. Please refer to:
https://ladsweb.modaps.eosdis.nasa.gov/ ... /?id=63313