wget Unable to establish SSL connection for MERRA2 data

[nwiseman]

Hi,
I am attempting to download subset MERRA2 data for a series of files (n = 214) and have been having issues with downloading subset data due to the destination name being too long. I am running on a windows machine with wget v1.21.31 (in my Windows/System32 directory). My input is the following:
> wget --load-cookies C:\Users\niciw\OneDrive\Documents\MATLAB\ocean_color\data\MERRA2\.urs_cookies --save-cookies C:\Users\niciw\OneDrive\Documents\MATLAB\ocean_color\data\MERRA2\.urs_cookies --auth-no-challenge=on --keep-session-cookies --user=nwiseman --ask-password --content-disposition -i MERRA2_Atlantic.txt
where MERRA2_Atlantic.txt is my list of files from GES DISC Data Access.
For example, one file name is: https://goldsmr4.gesdisc.eosdis.nasa.gov/opendap/MERRA2_DIURNAL/M2TUNXADG.5.12.4/2006/MERRA2_300.tavgU_2d_adg_Nx.200602.nc4.nc4?BCDP001[0:23][180:250][192:272],BCDP002[0:23][180:250][192:272],BCWT001[0:23][180:250][192:272],BCWT002[0:23][180:250][192:272],DUDP001[0:23][180:250][192:272],DUDP002[0:23][180:250][192:272],DUDP003[0:23][180:250][192:272],DUDP004[0:23][180:250][192:272],DUDP005[0:23][180:250][192:272],DUWT001[0:23][180:250][192:272],DUWT002[0:23][180:250][192:272],DUWT003[0:23][180:250][192:272],DUWT004[0:23][180:250][192:272],DUWT005[0:23][180:250][192:272],time,lat[180:250],lon[192:272]
When I run the command, the initial file (the README.pdf) downloads with no issues, and then as soon as it gets to the next file I get the following:
https://goldsmr4.gesdisc.eosdis.nasa.gov/opendap/MERRA2_DIURNAL/M2TUNXADG.5.12.4/2006/MERRA2_300.tavgU_2d_adg_Nx.200602.nc4.nc4?BCDP001[0:23][180:250][192:272],BCDP002[0:23][180:250][192:272],BCWT001[0:23][180:250][192:272],BCWT002[0:23][180:250][192:272],DUDP001[0:23][180:250][192:272],DUDP002[0:23][180:250][192:272],DUDP003[0:23][180:250][192:272],DUDP004[0:23][180:250][192:272],DUDP005[0:23][180:250][192:272],DUWT001[0:23][180:250][192:272],DUWT002[0:23][180:250][192:272],DUWT003[0:23][180:250][192:272],DUWT004[0:23][180:250][192:272],DUWT005[0:23][180:250][192:272],time,lat[180:250],lon[192:272]
Connecting to goldsmr4.gesdisc.eosdis.nasa.gov (goldsmr4.gesdisc.eosdis.nasa.gov)|198.118.197.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
The destination name is too long (608), reducing to 241
Location: https://urs.earthdata.nasa.gov/oauth/authorize/?scope=uid&app_type=401&client_id=e2WVk8Pw6weeLUKZYOxvTQ&response_type=code&redirect_uri=http%3A%2F%2Fgoldsmr4.gesdisc.eosdis.nasa.gov%2Fdata-redirect&state=aHR0cHM6Ly9nb2xkc21yNC5nZXNkaXNjLmVvc2Rpcy5uYXNhLmdvdi9vcGVuZGFwL01FUlJBMl9ESVVSTkFML00yVFVOWEFERy41LjEyLjQvMjAwNi9NRVJSQTJfMzAwLnRhdmdVXzJkX2FkZ19OeC4yMDA2MDIubmM0Lm5jND9CQ0RQMDAxWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLEJDRFAwMDJbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sQkNXVDAwMVswOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxCQ1dUMDAyWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVRFAwMDFbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sRFVEUDAwMlswOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxEVURQMDAzWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVRFAwMDRbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sRFVEUDAwNVswOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxEVVdUMDAxWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVV1QwMDJbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sRFVXVDAwM1swOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxEVVdUMDA0WzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVV1QwMDVbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sdGltZSxsYXRbMTgwOjI1MF0sbG9uWzE5MjoyNzJd [following]
--2023-03-03 14:45:36-- https://urs.earthdata.nasa.gov/oauth/authorize/?scope=uid&app_type=401&client_id=e2WVk8Pw6weeLUKZYOxvTQ&response_type=code&redirect_uri=http%3A%2F%2Fgoldsmr4.gesdisc.eosdis.nasa.gov%2Fdata-redirect&state=aHR0cHM6Ly9nb2xkc21yNC5nZXNkaXNjLmVvc2Rpcy5uYXNhLmdvdi9vcGVuZGFwL01FUlJBMl9ESVVSTkFML00yVFVOWEFERy41LjEyLjQvMjAwNi9NRVJSQTJfMzAwLnRhdmdVXzJkX2FkZ19OeC4yMDA2MDIubmM0Lm5jND9CQ0RQMDAxWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLEJDRFAwMDJbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sQkNXVDAwMVswOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxCQ1dUMDAyWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVRFAwMDFbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sRFVEUDAwMlswOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxEVURQMDAzWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVRFAwMDRbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sRFVEUDAwNVswOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxEVVdUMDAxWzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVV1QwMDJbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sRFVXVDAwM1swOjIzXVsxODA6MjUwXVsxOTI6MjcyXSxEVVdUMDA0WzA6MjNdWzE4MDoyNTBdWzE5MjoyNzJdLERVV1QwMDVbMDoyM11bMTgwOjI1MF1bMTkyOjI3Ml0sdGltZSxsYXRbMTgwOjI1MF0sbG9uWzE5MjoyNzJd
Resolving urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)... 198.118.243.33
Connecting to urs.earthdata.nasa.gov (urs.earthdata.nasa.gov)|198.118.243.33|:443... connected.
Unable to establish SSL connection.
I can download the individual files manually but it takes quite a long time due to the number of files. Any help would be greatly appreciated!

[GES DISC - mgreene]

Check if you can download the file https://goldsmr4.gesdisc.eosdis.nasa.gov/data/MERRA2_MONTHLY/M2TMNXRAD.5.12.4/1980/MERRA2_100.tavgM_2d_rad_Nx.198001.nc4 from the Web browser on your Windows desktop, if you do, then verify that you enter the correct password in wget prompt.
If the above steps check out then most likely the wget is outdated, check its version and see if you can install wget 1.21.

Data access instruction can be found here, https://disc.gsfc.nasa.gov/data-access

[michelleescobarcarias01]

I am having a similar issue but in my case, I first run into the following error message:
wget: unrecognised option `--ask-password'

The command I'm using is:
wget --load-cookies C:\Users\mesc0003\MERRA\.urs_cookies --save-cookies C:\Users\mesc0003\MERRA\.urs_cookies --auth-no-challenge=on --keep-session-cookies --user=michelleescobarcarias01 --ask-password --content-disposition -i \\ad.monash.edu\home\User042\mesc0003\Documents\NASA\nasa_humidity.txt

Any help would be highly appreciated

[michelleescobarcarias01]

So, I managed to solve the issue by updating my wget version but now, when I run the code only 2 documents are downloaded: the readme.pdf and the first link (day) of data. Then it stops as if it is done. Any clues?

[GES DISC - mgreene]

Re: wget Unable to establish SSL connection for MERRA2 data

by michelleescobarcarias01 » Mon Mar 06, 2023 11:07 pm America/New_York
So, I managed to solve the issue by updating my wget version but now, when I run the code only 2 documents are downloaded: the readme.pdf and the first link (day) of data. Then it stops as if it is done. Any clues?
Top

There was a server upgrade that may have cause this to happen. Please try again and let us know the status. thanks

[michelleescobarcarias01]

Hi,

I tried again several times but the same keeps happening.
Any recommendations?

Thanks!

Michelle

[GES DISC - mgreene]

Could you please provide us with the wget version you use now. We would also like for you to try the following
command, thanks:

wget --load-cookies C:\.urs_cookies --save-cookies C:\.urs_cookies
--auth-no-chal
lenge=on --keep-session-cookies --user=michelleescobarcarias01 --ask-password
--content-disposition
"https://goldsmr4.gesdisc.eosdis.nasa.gov/opendap/MERRA2_DIURNAL/M2TUNXADG.5.12.4/2006/MERRA2_300.tavgU_2d_adg_Nx.200602.nc4.nc4?BCDP001[0:23][180:250][192:272],BCDP002[0:23][180:250][192:272],BCWT001[0:23][180:250][192:272],BCWT002[0:23][180:250][192:272],DUDP001[0:23][180:250][192:272],DUDP002[0:23][180:250][192:272],DUDP003[0:23][180:250][192:272],DUDP004[0:23][180:250][192:272],DUDP005[0:23][180:250][192:272],DUWT001[0:23][180:250][192:272],DUWT002[0:23][180:250][192:272],DUWT003[0:23][180:250][192:272],DUWT004[0:23][180:250][192:272],DUWT005[0:23][180:250][192:272],time,lat[180:250],lon[192:272]"


If the one-line command work, then put this command line into a loop to loop
through a time range using a programming language such as cshell.

[GES DISC - mgreene]

Following up to see if the information provided met your needs.