Hi,
I am trying to download a MODIS .hdf file from LAADS using curl with deliberately incorrect credentials, meaning the below username and password are mere placeholders. Running the following command
curl \
-u "sad_boyd:Lc557Gv$" \
--fail \
-sS \
-L \
-O https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MCD64A1/2019/335/MCD64A1.A2019335.h32v11.061.2021309110404.hdf
creates a file MCD64A1.A2019335.h32v11.061.2021309110404.hdf in my current directory, but the file content is actually the "please login" HTML web page. By contrast, running the LP DAAC analog
curl \
-u "sad_boyd:Lc557Gv$" \
--fail \
-sS \
-L \
-O https://e4ftl01.cr.usgs.gov/MOTA/MCD64A1.061/2019.12.01/MCD64A1.A2019335.h32v11.061.2021309110404.hdf
raises an authentication error "curl: (22) The requested URL returned error: 401" and does not create any file, which I think is the more error-prone strategy.
I am not entirely certain, but didn't the above curl command for LAADS raise an error until a while ago, just like it still does for LP DAAC? Is it somehow possible to force curl to raise an error instead of downloading the HTML web content from LAADS when authentication fails? For the record, the same happens when using wget instead of curl.
curl download with incorrect credentials writes HTML content to HDF
Re: curl download with incorrect credentials writes HTML content to HDF
Adding a `-v` flag to see what happens, it looks like the first URL returns a '200 OK' status with the error HTML, rather than the correct '401' response code. I don't think you can have curl fail easily here, as the server is returning the wrong code.
You can use the `file` command on the downloaded file to see if it really is hdf5. If it's HTML, file will return something like `MCD64A1.A2019335.h32v11.061.2021309110404.hdf: HTML document text, ASCII text, with very long lines (460)` instead
You can use the `file` command on the downloaded file to see if it really is hdf5. If it's HTML, file will return something like `MCD64A1.A2019335.h32v11.061.2021309110404.hdf: HTML document text, ASCII text, with very long lines (460)` instead
Re: curl download with incorrect credentials writes HTML content to HDF
Or you can useYou can use the `file` command on the downloaded file to see if it really is hdf5
Code: Select all
wget —adjust-extension