Page 1 of 1

ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Mon Mar 08, 2021 12:52 pm America/New_York
by srleone9
Novice coder here, read: Windows 10 user who knows what a DOS prompt is, lol
...
I am wracking my brain trying to use the Cygwin console to download many, many thousands of files to add to a GIS (using QGIS).
I have successfully downloaded the shell file script (download.sh) and after some digging in the Cygwin user guide, figured out where to save it to take advantage of Cygwin's default file paths.
However, it seems no matter how many applications I authorize in my Earthdata profile, after entering my Earthdata login credentials, I get the same message that refers to the same URL, which is just an all-black tif file.
Here's the error message itself:
Unable to Retrieve Data
Please ensure that you have authorized the remote application by visiting the link below
https://lpdaac.earthdata.nasa.gov/lp-pr ... 80_dem.tif

I found directions on authorizing apps at https://wiki.earthdata.nasa.gov/display ... pplication
I looked at the download script itself in Notepad, and it appears that the error message is not dynamic, so methinks it's not really a useful URL...I mined the script file for letter combos in applications to authorize but, as I said, no change in the error message.
...
What is the magic remote application that must be authorized?
How do I get my data?!? HELP please :-)

Thanks, Su

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Mon Mar 08, 2021 3:12 pm America/New_York
by LP DAAC - jwilson
After you create your account, you will also need to “authorize” the LP DAAC Data Pool application. Once the Data Pool application has been authorized, you will be able to download data.

Please let me know if you continue to have issues.
Kind regards,
Janice

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Mon Mar 08, 2021 4:00 pm America/New_York
by srleone9
Janice, that was the first app I authorized, since it's mentioned in that old wiki article.
Hence, my absolute confusion. I ended up downloading ETOPO1, but the resolution is not as good.
Any other pointers?
Thanks,
Su

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Tue Mar 09, 2021 10:56 am America/New_York
by LP DAAC - jwilson
Dear Su,
We searched the Earthdata Search client for the granule ASTGTMV003N36E080.zip and downloaded it with no issues. We could not duplicate your issue with the data not being available. Please see attachment.

LP DAAC does not support software and I'm not familiar with the Cygwin software. You might need to contact their support for further guidance.

I hope this helps,
Kind regards,
Janice

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Tue Mar 09, 2021 11:48 am America/New_York
by srleone9
Downloading one at a time is not the problem, nor is that specific granule.
Cygwin is what Earthdata suggests for downloading using a script. Otherwise it would take a year. Literally.
For some reason, that granule is hard-coded into the script generated by Earthdata as the output for all "unable to retrieve" error messages.

Any ideas?

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Wed Mar 10, 2021 5:02 pm America/New_York
by LP DAAC - jwilson
Hi Su,
You might benefit from using the AppEEARS tool.

AppEEARS landing page: https://lpdaacsvc.cr.usgs.gov/appeears/
You will need to use your Earthdata Login
Getting started instructions can be found here: https://lpdaacsvc.cr.usgs.gov/appeears/help

I'm going to forward your question to our subject matter expert for further guidance.
Thank you for your patience.
Janice

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Thu Mar 11, 2021 3:23 am America/New_York
by roblox_free_robux
Hi Mate,

Thank You, For Sharing The Information and Really Appreciated

Re: ASTER Data Download Cygwin Unable to Retrieve Data

Posted: Thu Mar 11, 2021 11:48 am America/New_York
by LP DAAC - ckrehbiel
@@srleone9 I unfortunately can't help with the cygwin issue, for that specifically, I would defer to: support@earthdata.nasa.gov

However, it looks like your use case is that you'd like to bulk download many different ASTER GDEM files directly from the LP DAAC Data Pool? There are a couple of solutions I can point you towards.
First, once you have the desired files in Earthdata Search, ex: https://search.earthdata.nasa.gov/proje ... 99669636!4!!

You will see this page:
2021-03-11_10-41-21.png
2021-03-11_10-41-21.png (34.79 KiB) Not viewed yet
Go ahead and save the list of links as a text file. Next, we have multiple different options that will allow you to bulk download these files, including:

How to bulk download from the command line using wget or curl: https://lpdaac.usgs.gov/resources/e-lea ... mand-line/

How to Access the LP DAAC Data Pool with Python: https://git.earthdata.nasa.gov/projects ... hon/browse

How to Access the LP DAAC Data Pool with R:
https://git.earthdata.nasa.gov/projects ... d_r/browse

You should be able to feed that text file (using the instructions provided for each of the options) into any of the options listed above in order to bulk download your desired files.

Let me know if you are still encountering issues after trying these options.

Thanks!
Cole