Earthdata Forum

I'm using WGET to download L2 files to my server.  The files are showing up but seem to be capped at 13 kb.  Any idea why?

WGET Command
wget -P ~/www/saltwx.com/public_html/MODIS/sst/ --user=my-user--password='my-pw' --auth-no-challenge=on "https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png"

Result

--2020-02-17 02:51:01--  https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA [following]
--2020-02-17 02:51:01--  https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA
Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:241a:4081::89
Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: '/home/my_key/www/saltwx.com/public_html/MODIS/sst/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png'

     0K .......... ..                                           381K=0.03s

2020-02-17 02:51:01 (381 KB/s) - '/home/my_key/www/saltwx.com/public_html/MODIS/sst/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png'
saved [12671] <----This appears to be the total file size downloaded.

This usually indicates that an HTML file was downloaded.  If you're on a Linux platform, you can try the 'file' command on the saved file.  If it is HTML, you should be able to view it with a text viewer or in a browser.  A lot of times this ends up being the Earthdata login page and might mean your login credentials are not being accepted.

john

Rather than a partial download you may getting an html document you can view in a browser.   Try adding the wget --adjust-extension option.   You may need the --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies options.

So, like this?

wget -P ~/www/saltwx.com/public_html/MODIS/sst/ --adjust-extension --user=user--password='password' --auth-no-challenge=on "https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png"

OR

wget -P ~/www/saltwx.com/public_html/MODIS/sst/ --adjust-extension --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies "https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png"

Postscript: I able was to open the file and it is indeed the login page.  I'm beginning to feel that there is no way to simply download a PNG file with WGET.

I've tried all suggestions and just cannot get the single file I would like, to actually download.   

Are there any other suggestions to how to get this file : A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png to download non-interactively?

All that I am getting is the login page html downloaded to my server.

Wget (as well as wget2) also fails for me, but downloads succeed using a browser or the following curl incantation (from Download Methods):

curl -O -b ~/.urs_cookies -c ~/.urs_cookies -L -n https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png

here are the results of both methods and there appears to be no difference in outcome.  Still only getting the HTML file

1
Command
wget -P ~/www/saltwx.com/public_html/MODIS/sst/ --user=user--password='PW' --auth-no-challenge=on "https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png"

Result
--2020-02-17 18:50:01--  https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&required_scope=study_area,country [following]
--2020-02-17 18:50:02--  https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&required_scope=study_area,country
Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:241a:4081::89
Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: '/home/u250-zd3apyn5m53e/www/saltwx.com/public_html/MODIS/sst/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png.2'

     0K .......... ..                                          41.1M=0s

2020-02-17 18:50:02 (41.1 MB/s) - '/home/u250-zd3apyn5m53e/www/saltwx.com/public_html/MODIS/sst/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png.2' saved [12859]

2
Command
wget -P ~/www/saltwx.com/public_html/MODIS/sst/ --user=user--password='pw' --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --auth-no-challenge=on "https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png"

Result
--2020-02-17 18:40:01--  https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&required_scope=study_area,country [following]
--2020-02-17 18:40:08--  https://urs.earthdata.nasa.gov/oauth/authorize?redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict&response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&required_scope=study_area,country
Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:241a:4081::89
Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: '/home/u250-zd3apyn5m53e/www/saltwx.com/public_html/MODIS/sst/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png.1'

     0K .......... ..                                          56.3M=0s

2020-02-17 18:40:08 (56.3 MB/s) - '/home/u250-zd3apyn5m53e/www/saltwx.com/public_html/MODIS/sst/A2020046181000.L2_LAC.S3470.nc.Southeast.sst.png.1' saved [12859]

OK, the cURL seemed to download but I think it is still only delivering the HTML page.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 12859    0 12859    0     0  38157      0 --:--:-- --:--:-- --:--:-- 38157

Confirmed that each method, WGET and cURL, result in a 12.859 KB file download. 

This authentication process is ridiculous.

The cURL method requires a .netrc file (wget can also use this file, but you will need the --auth-no-challenge=on option) that contains your login Earthdata credentials.
Verify you have one, and it is correct, or generate one ala:

echo "machine urs.earthdata.nasa.gov login USERNAME password PASSWD" > ~/.netrc
chmod  0600 ~/.netrc

https://oceancolor.gsfc.nasa.gov/data/download_methods/ describes the process
Sean