I need to batch download MYD06_L2 and MYD03 files over a specific geographic region. I have done this many times in the past, but it's been about a year since my last attempt. My bearer token is currently valid. I am unable to get *any* scripts (either my own or the templates provided by LAADS DAAC tutorials) to complete a download.
Using the LAADS DAAC tutorial examples:
curl -v -H 'Authorization: Bearer $TOKEN' 'https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD03/2011/121/MYD03.A2011121.2100.061.2018029063924.hdf' > ./MYD03.A2011121.2100.061.2018029063924.hdf
wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD03/2011/121/MYD03.A2011121.2100.061.2018029063924.hdf" --header "Authorization: Bearer $TOKEN" -P ./
I either get "504 Gateway Time-out" or "401 Username/Password Authentication failed" errors (only timeouts with curl, both errors with wget - first a 504 then 401). It appears that wget is redirected to an authentication link (https://ladsweb.modaps.eosdis.nasa.gov/oauth/login?redirect=...), so maybe there's an added authentication step that I'm missing? Any help here would be greatly appreciated...
If you are using a valid token(generated from ladsweb) you don't need any other authentication to download data. Your wget and curl commands seems correct but probably you tried to download data at time when LAADS servers were very busy. For last several days we seem to get a spike every day starting at noon -- like someone’s big download script is starting and runs for an hour or two. The issue seems to be that the file server is slow. So, we are advising users that they should write their scripts to wait a bit then retry when they get errors. We cannot guarantee anything at this point.
In case you want to see if you are using the correct token then follow these steps:
1. Go to data folder:
For example, https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD03/2011/121/
2. "Login" at the top of the page.
3. Click on "See wget Download Command" tab, you will see complete wget command with your token already included.
LAADS User Services
To receive news from LAADS DAAC direct to your inbox, email firstname.lastname@example.org with “subscribe” in the subject line.
The data is subsetted in the Middle East, and I am trying to download only four tiles covering Iraq, which I mosaiced, converted to Gtiff format, then reprojected to Geographic. I submitted the order and received an email with a link to the data and a wget script; I followed the instructions in the email and added a new token.
The issue is that wget cannot access the files because it receives a 404 ERROR (.tif file not available for download), despite the fact that wget can reach your server since it downloaded the README GLDAS2.pdf file.
Please keep in mind that I attempted to download them manually by following the instructions, but it did not work and I had the same issue. Finally, I resubmitted the order, but I ran into the same issue, which I assume is due to the fact that the data I downloaded there is not geographically subset and is not in Gtiff format.
I've included a cmd window to give you a better idea of the situation.
Thank you for your consideration. I eagerly await your response!
- LAAD DAAC issue.PNG (113.34 KiB) Not viewed yet