Page 1 of 1

Download a File with cURL or Wget

Posted: Mon May 24, 2021 11:35 am America/New_York
by njester
The following allows you to download data from https://asdc.larc.nasa.gov/data/ using linux, a mac, or Windows with Cygin.

1) If you haven't before, create an authentication cookie that can be used with to access files behind the EarthData login page.

Code: Select all

USERNAME=<your earthdata username>
PASSWORD=<your earthdata password>
cd ~ 
touch .netrc 
echo "machine urs.earthdata.nasa.gov login $USERNAME password $PASSWORD" > .netrc 
chmod 0600 .netrc 
touch .urs_cookies 
2) You can download data with the following commands:

Code: Select all

URL=<your url here>
curl -b ~/.urs_cookies -c ~/.urs_cookies --remote-name --location  --netrc $URL
or

Code: Select all

wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --auth-no-challenge=on --keep-session-cookies --content-disposition $URL
You can learn more about these command line tools here:
https://www.gnu.org/software/wget/
https://curl.se/docs/
https://wiki.earthdata.nasa.gov/display ... L+And+Wget

Re: Download a File with cURL or Wget

Posted: Sat Feb 19, 2022 2:16 pm America/New_York
by danpeterson
Filename Is Too Long Sometimes you get an error when trying to move, delete, or copy directories where the character count for the file path is more than 260.

Note that the words directory and folder are interchangeable. We’ll use ‘directory’ going forward. The following PowerShell cmdlets can also be used on files.

That file path is 280 characters long. So we cannot copy the directory out of there to somewhere else with the normal copy-paste method. We get the Destination Path Too Long error.

Re: Download a File with cURL or Wget

Posted: Tue Mar 01, 2022 11:25 am America/New_York
by njester
danpeterson wrote: Sat Feb 19, 2022 2:16 pm America/New_York Filename Is Too Long Sometimes you get an error when trying to move, delete, or copy directories where the character count for the file path is more than 260.

Note that the words directory and folder are interchangeable. We’ll use ‘directory’ going forward. The following PowerShell cmdlets can also be used on files.

That file path is 280 characters long. So we cannot copy the directory out of there to somewhere else with the normal copy-paste method. We get the Destination Path Too Long error.
Could you provide the names or links of some of the files that are too long?