Wget download error

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
zcwangyu
Posts: 1
Joined: Mon Aug 26, 2024 10:39 pm America/New_York
Answers: 0

Wget download error

by zcwangyu » Mon Aug 26, 2024 10:48 pm America/New_York

Hi, I am trying to use wget to download the MOD35_L2 data, and the command is: wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD35_L2/2021//" --header "Authorization: Bearer INSERT_EDL_TOKEN_HERE" -P . But it failed with the tips as following:
--2024-08-27 10:44:00-- https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD35_L2/2021//
Resolving ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)... 198.118.194.40
Connecting to ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)|198.118.194.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 246662 (241K) [text/html]
Saving to: './MOD35_L2/2021/index.html'

MOD35_L2/2021/index 100%[===================>] 240.88K 247KB/s in 1.0s

Last-modified header missing -- time-stamps turned off.
2024-08-27 10:44:04 (247 KB/s) - './MOD35_L2/2021/index.html' saved [246662/246662]

FINISHED --2024-08-27 10:44:04--
Total wall clock time: 3.5s
Downloaded: 1 files, 241K in 1.0s (247 KB/s)

I tried the command: wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD35_L2/2021/" --header "Authorization: Bearer INSERT_EDL_TOKEN_HERE" -P . It also failed.
--2024-08-27 10:33:34-- https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD35_L2/2021/
Resolving ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)... 198.118.194.40
Connecting to ladsweb.modaps.eosdis.nasa.gov (ladsweb.modaps.eosdis.nasa.gov)|198.118.194.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 246662 (241K) [text/html]
Saving to: './MOD35_L2/2021/index.html'

MOD35_L2/2021/index 100%[===================>] 240.88K 339KB/s in 0.7s

Last-modified header missing -- time-stamps turned off.
2024-08-27 10:33:37 (339 KB/s) - './MOD35_L2/2021/index.html' saved [246662/246662]

--2024-08-27 10:33:37-- https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD35_L2/2021/001/
Reusing existing connection to ladsweb.modaps.eosdis.nasa.gov:443.
HTTP request sent, awaiting response... 200 OK
Length: 415380 (406K) [text/html]
Saving to: './MOD35_L2/2021/001/index.html'

MOD35_L2/2021/001/i 100%[===================>] 405.64K 421KB/s in 1.0s

Last-modified header missing -- time-stamps turned off.
2024-08-27 10:33:40 (421 KB/s) - './MOD35_L2/2021/001/index.html' saved [415380/415380]

But!!! I can download the README text file, and download data using python script. I am trying to use wget command for downloading data. Please help me! Please Please~

Filters:

LAADS-EDL_UserServices_M
Posts: 396
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 2
Has endorsed: 5 times

Re: Wget download error

by LAADS-EDL_UserServices_M » Tue Aug 27, 2024 8:35 am America/New_York

There is a typo in the data link in the wget command on our website. In the wget command on the page it shows two forward slashes "//" at the end of the URL where there should be only one, see:
https://ladsweb.modaps.eosdis.nasa.gov/alerts-and-issues/148498
The command will work if you remove the extra slash like:
wget -e robots=off -m -np -R .html,.tmp -nH --cut-dirs=3 "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD35_L2/2021/" --header "Authorization: Bearer INSERT_EDL_TOKEN_HERE" -P .
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

Post Reply