Page 1 of 1

Wget Script to Use For Direct Data Dowload Data

Posted: Wed Jul 22, 2020 12:06 pm America/New_York
by EarthdataForumContributor
I was interested in downloading CAPLIPSO expedited data from the ASDC’s direct data download. There’s some useful information on how to use scripts to download the data, but when I tried using the wget script in the example, I’m only able to get a specific file. I was wondering if it was possible to download the entire contents of a directory, rather than individual files?

Re: Wget Script to Use For Direct Data Dowload Data

Posted: Wed Jul 22, 2020 12:21 pm America/New_York
by ASDC - mcook
Here is an example of a Wget script for downloading the entire directory.
Just substitute your product data set in place of the example (CALIPSO data set) below.

URL= https://asdc.larc.nasa.gov/data/CALIPSO/LID_L1_Exp-Prov-V3-40/
USERNAME=<your Earthdata login username>
PASSWORD=<your Earthdata login password>
cd ~
touch .netrc
echo "machine urs.earthdata.nasa.gov login $USERNAME password $PASSWORD" > .netrc
chmod 0600 .netrc
touch .urs_cookies
wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --auth-no-challenge=on --keep-session-cookies --content-disposition --recursive --wait=60 --no-parent --reject "index.html*" --execute robots=off $URL