curl download with incorrect credentials writes HTML content to HDF
Posted: Wed Jan 04, 2023 4:09 am America/New_York
Hi,
I am trying to download a MODIS .hdf file from LAADS using curl with deliberately incorrect credentials, meaning the below username and password are mere placeholders. Running the following command
curl \
-u "sad_boyd:Lc557Gv$" \
--fail \
-sS \
-L \
-O https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MCD64A1/2019/335/MCD64A1.A2019335.h32v11.061.2021309110404.hdf
creates a file MCD64A1.A2019335.h32v11.061.2021309110404.hdf in my current directory, but the file content is actually the "please login" HTML web page. By contrast, running the LP DAAC analog
curl \
-u "sad_boyd:Lc557Gv$" \
--fail \
-sS \
-L \
-O https://e4ftl01.cr.usgs.gov/MOTA/MCD64A1.061/2019.12.01/MCD64A1.A2019335.h32v11.061.2021309110404.hdf
raises an authentication error "curl: (22) The requested URL returned error: 401" and does not create any file, which I think is the more error-prone strategy.
I am not entirely certain, but didn't the above curl command for LAADS raise an error until a while ago, just like it still does for LP DAAC? Is it somehow possible to force curl to raise an error instead of downloading the HTML web content from LAADS when authentication fails? For the record, the same happens when using wget instead of curl.
I am trying to download a MODIS .hdf file from LAADS using curl with deliberately incorrect credentials, meaning the below username and password are mere placeholders. Running the following command
curl \
-u "sad_boyd:Lc557Gv$" \
--fail \
-sS \
-L \
-O https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MCD64A1/2019/335/MCD64A1.A2019335.h32v11.061.2021309110404.hdf
creates a file MCD64A1.A2019335.h32v11.061.2021309110404.hdf in my current directory, but the file content is actually the "please login" HTML web page. By contrast, running the LP DAAC analog
curl \
-u "sad_boyd:Lc557Gv$" \
--fail \
-sS \
-L \
-O https://e4ftl01.cr.usgs.gov/MOTA/MCD64A1.061/2019.12.01/MCD64A1.A2019335.h32v11.061.2021309110404.hdf
raises an authentication error "curl: (22) The requested URL returned error: 401" and does not create any file, which I think is the more error-prone strategy.
I am not entirely certain, but didn't the above curl command for LAADS raise an error until a while ago, just like it still does for LP DAAC? Is it somehow possible to force curl to raise an error instead of downloading the HTML web content from LAADS when authentication fails? For the record, the same happens when using wget instead of curl.