Inability to Download OB.DAAC Data with obdaac_download.py
-
- Posts: 6
- Joined: Thu Oct 19, 2023 1:28 pm America/New_York
Inability to Download OB.DAAC Data with obdaac_download.py
Using the python method provided in the data download methods (https://oceancolor.gsfc.nasa.gov/data/download_methods/), I am trying to download many day files of SNPP-VIIRS Rrs data. I am working on a machine using Windows 10 and my current python version is 3.8.10.
Without changing the py file I issue this command to the script:
python obdaac_download.py -v --filelist C:\Users\path\to\file\list --odir C:\Users\path\to\output\location --appkey myappkey
Output directory: C:\Users\path\to\output\location
Retrieving https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/SNPP_VIIRS.20200101.L3m.DAY.RRS.Rrs_410.4km.nc
and after trying to reach the url to download the data the following error is returned:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='oceandata.sci.gsfc.nasa.gov', port=443): Max retries exceeded with url: /cgi/getfile/SNPP_VIIRS.20200101.L3m.DAY.RRS.Rrs_410.4km.nc?appkey=myappkey (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000022CA4F261C0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))
Is there a known solution to this error or is there something else that I need to do on my system to allow for it to grab data remotely?
Without changing the py file I issue this command to the script:
python obdaac_download.py -v --filelist C:\Users\path\to\file\list --odir C:\Users\path\to\output\location --appkey myappkey
Output directory: C:\Users\path\to\output\location
Retrieving https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/SNPP_VIIRS.20200101.L3m.DAY.RRS.Rrs_410.4km.nc
and after trying to reach the url to download the data the following error is returned:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='oceandata.sci.gsfc.nasa.gov', port=443): Max retries exceeded with url: /cgi/getfile/SNPP_VIIRS.20200101.L3m.DAY.RRS.Rrs_410.4km.nc?appkey=myappkey (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000022CA4F261C0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))
Is there a known solution to this error or is there something else that I need to do on my system to allow for it to grab data remotely?
Filters:
-
- Subject Matter Expert
- Posts: 450
- Joined: Fri Feb 05, 2021 9:17 am America/New_York
- Been thanked: 7 times
Re: Inability to Download OB.DAAC Data with obdaac_download.py
What happens if you paste the URL into a web browser?
https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/SNPP_VIIRS.20200101.L3m.DAY.RRS.Rrs_410.4km.nc
Tommy
https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/SNPP_VIIRS.20200101.L3m.DAY.RRS.Rrs_410.4km.nc
Tommy
-
- Posts: 6
- Joined: Thu Oct 19, 2023 1:28 pm America/New_York
Re: Inability to Download OB.DAAC Data with obdaac_download.py
It downloads the file just as if I went through the direct data access repository.
-
- Subject Matter Expert
- Posts: 450
- Joined: Fri Feb 05, 2021 9:17 am America/New_York
- Been thanked: 7 times
Re: Inability to Download OB.DAAC Data with obdaac_download.py
and the download script is still timing out on the file you transferred via the browser?
Tommy
Tommy
-
- Posts: 6
- Joined: Thu Oct 19, 2023 1:28 pm America/New_York
Re: Inability to Download OB.DAAC Data with obdaac_download.py
Correct. I just tried again this morning using both the single file input to the script and using the filelist methods and I received the same error again.
-
- Posts: 1519
- Joined: Wed Sep 18, 2019 6:15 pm America/New_York
- Been thanked: 9 times
Re: Inability to Download OB.DAAC Data with obdaac_download.py
Eric,
I verified the script on Windows10. I was able to retrieve the file you report as an example (obviously I used my own appkey). The error you're seeing suggests a problem with the connection - the script is trying to connect, but cannot. Odd that you can get the file from the browser, but not the script. Do you have a local firewall preventing web connections from python scripts? Grasping at straws here...
Sean
I verified the script on Windows10. I was able to retrieve the file you report as an example (obviously I used my own appkey). The error you're seeing suggests a problem with the connection - the script is trying to connect, but cannot. Odd that you can get the file from the browser, but not the script. Do you have a local firewall preventing web connections from python scripts? Grasping at straws here...
Sean
-
- Posts: 6
- Joined: Thu Oct 19, 2023 1:28 pm America/New_York
Re: Inability to Download OB.DAAC Data with obdaac_download.py
I am working on a DoD computer, so it is possible that we have some protective measures from pulling things online. I will reach out to my IT department to see if they have a better understanding on what is blocking this download process through the script.