ASTER Data Download Cygwin Unable to Retrieve Data

Please enter here to ask a question about any NASA Science related topics!
Post Reply
srleone9
Posts: 3
Joined: Mon Mar 08, 2021 12:34 pm America/New_York

ASTER Data Download Cygwin Unable to Retrieve Data

by srleone9 » Mon Mar 08, 2021 12:52 pm America/New_York

Novice coder here, read: Windows 10 user who knows what a DOS prompt is, lol
...
I am wracking my brain trying to use the Cygwin console to download many, many thousands of files to add to a GIS (using QGIS).
I have successfully downloaded the shell file script (download.sh) and after some digging in the Cygwin user guide, figured out where to save it to take advantage of Cygwin's default file paths.
However, it seems no matter how many applications I authorize in my Earthdata profile, after entering my Earthdata login credentials, I get the same message that refers to the same URL, which is just an all-black tif file.
Here's the error message itself:
Unable to Retrieve Data
Please ensure that you have authorized the remote application by visiting the link below
https://lpdaac.earthdata.nasa.gov/lp-pr ... 80_dem.tif

I found directions on authorizing apps at https://wiki.earthdata.nasa.gov/display ... pplication
I looked at the download script itself in Notepad, and it appears that the error message is not dynamic, so methinks it's not really a useful URL...I mined the script file for letter combos in applications to authorize but, as I said, no change in the error message.
...
What is the magic remote application that must be authorized?
How do I get my data?!? HELP please :-)

Thanks, Su

Tags:

LP DAAC - jwilson
User Services
User Services
Posts: 56
Joined: Mon Sep 30, 2019 12:39 pm America/New_York
Answers: 1

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by LP DAAC - jwilson » Mon Mar 08, 2021 3:12 pm America/New_York

After you create your account, you will also need to “authorize” the LP DAAC Data Pool application. Once the Data Pool application has been authorized, you will be able to download data.

Please let me know if you continue to have issues.
Kind regards,
Janice

srleone9
Posts: 3
Joined: Mon Mar 08, 2021 12:34 pm America/New_York

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by srleone9 » Mon Mar 08, 2021 4:00 pm America/New_York

Janice, that was the first app I authorized, since it's mentioned in that old wiki article.
Hence, my absolute confusion. I ended up downloading ETOPO1, but the resolution is not as good.
Any other pointers?
Thanks,
Su

LP DAAC - jwilson
User Services
User Services
Posts: 56
Joined: Mon Sep 30, 2019 12:39 pm America/New_York
Answers: 1

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by LP DAAC - jwilson » Tue Mar 09, 2021 10:56 am America/New_York

Dear Su,
We searched the Earthdata Search client for the granule ASTGTMV003N36E080.zip and downloaded it with no issues. We could not duplicate your issue with the data not being available. Please see attachment.

LP DAAC does not support software and I'm not familiar with the Cygwin software. You might need to contact their support for further guidance.

I hope this helps,
Kind regards,
Janice
Attachments
ASTGTMV003_N36E080.jpg
ASTGTMV003_N36E080.jpg (46.04 KiB) Not viewed yet

srleone9
Posts: 3
Joined: Mon Mar 08, 2021 12:34 pm America/New_York

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by srleone9 » Tue Mar 09, 2021 11:48 am America/New_York

Downloading one at a time is not the problem, nor is that specific granule.
Cygwin is what Earthdata suggests for downloading using a script. Otherwise it would take a year. Literally.
For some reason, that granule is hard-coded into the script generated by Earthdata as the output for all "unable to retrieve" error messages.

Any ideas?

LP DAAC - jwilson
User Services
User Services
Posts: 56
Joined: Mon Sep 30, 2019 12:39 pm America/New_York
Answers: 1

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by LP DAAC - jwilson » Wed Mar 10, 2021 5:02 pm America/New_York

Hi Su,
You might benefit from using the AppEEARS tool.

AppEEARS landing page: https://lpdaacsvc.cr.usgs.gov/appeears/
You will need to use your Earthdata Login
Getting started instructions can be found here: https://lpdaacsvc.cr.usgs.gov/appeears/help

I'm going to forward your question to our subject matter expert for further guidance.
Thank you for your patience.
Janice

roblox_free_robux
Posts: 1
Joined: Tue Mar 09, 2021 12:59 am America/New_York

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by roblox_free_robux » Thu Mar 11, 2021 3:23 am America/New_York

Hi Mate,

Thank You, For Sharing The Information and Really Appreciated

LP DAAC - ckrehbiel
Subject Matter Expert
Subject Matter Expert
Posts: 59
Joined: Tue Nov 12, 2019 3:29 pm America/New_York

Re: ASTER Data Download Cygwin Unable to Retrieve Data

by LP DAAC - ckrehbiel » Thu Mar 11, 2021 11:48 am America/New_York

@@srleone9 I unfortunately can't help with the cygwin issue, for that specifically, I would defer to: support@earthdata.nasa.gov

However, it looks like your use case is that you'd like to bulk download many different ASTER GDEM files directly from the LP DAAC Data Pool? There are a couple of solutions I can point you towards.
First, once you have the desired files in Earthdata Search, ex: https://search.earthdata.nasa.gov/proje ... 99669636!4!!

You will see this page:
2021-03-11_10-41-21.png
2021-03-11_10-41-21.png (34.79 KiB) Not viewed yet
Go ahead and save the list of links as a text file. Next, we have multiple different options that will allow you to bulk download these files, including:

How to bulk download from the command line using wget or curl: https://lpdaac.usgs.gov/resources/e-lea ... mand-line/

How to Access the LP DAAC Data Pool with Python: https://git.earthdata.nasa.gov/projects ... hon/browse

How to Access the LP DAAC Data Pool with R:
https://git.earthdata.nasa.gov/projects ... d_r/browse

You should be able to feed that text file (using the instructions provided for each of the options) into any of the options listed above in order to bulk download your desired files.

Let me know if you are still encountering issues after trying these options.

Thanks!
Cole

Post Reply