SWOT_L2_HR_PIXC_2.0 scripting download

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
trevor.wilkerson
Posts: 5
Joined: Wed Aug 14, 2024 2:51 pm America/New_York
Answers: 0

SWOT_L2_HR_PIXC_2.0 scripting download

by trevor.wilkerson » Wed Aug 14, 2024 3:04 pm America/New_York

I am attempting to download SWOT_L2_HR_PIXC_2.0 via either the PODAAC data subscriber, or the python earthaccess package. In either case, the download is getting frozen and will not complete. I try the same script (for both podaac data subscriber and earthaccess) but for the SWOT_L2_HR_Raster_2.0 and find no issues. The query for the SWOT_L2_HR_PIXC_2.0 finds datasets and they start to download but never finish.

Has anyone else had issues with this data set?

CLI command used:
podaac-data-downloader -c SWOT_L2_HR_PIXC_2.0 -d tempfolder --start-date 2024-05-04T19:48:32Z --end-date 2024-05-14T19:48:43Z -b=-0.13698,51.48410,-0.04806,52.50708

No error messages, using a venv with podaac-data-subscriber installed, on a Windows 11.
Internet speeds of nearly 1gb/s on my end, attempted on both Wi-Fi and ethernet. Attempted on 2024-08-13 and 14. Any advice would be greatly appreciated.

Filters:

PODAAC - celiaoued
Subject Matter Expert
Subject Matter Expert
Posts: 46
Joined: Fri May 28, 2021 1:30 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by PODAAC - celiaoued » Wed Aug 14, 2024 7:51 pm America/New_York

Hello,
I wonder if you may have experienced a degraded service event. Would you be able to try again from the command line? Just now, I was able to run the exact same command on Windows and downloaded 6 files:
(base) C:\Projects\test>podaac-data-downloader -c SWOT_L2_HR_PIXC_2.0 -d tempfolder --start-date 2024-05-04T19:48:32Z --end-date 2024-05-14T19:48:43Z -b=-0.13698,51.48410,-0.04806,52.50708
[2024-08-14 16:37:35,102] {podaac_data_downloader.py:146} INFO - NOTE: Making new data directory at tempfolder(This is the first run.)
[2024-08-14 16:37:36,507] {podaac_data_downloader.py:317} INFO - Found 6 total files to download
[2024-08-14 16:38:14,842] {podaac_data_downloader.py:358} INFO - 2024-08-14 16:38:14.842035 SUCCESS: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_141_247L_20240513T084834_20240513T084845_PIC0_01.nc
[2024-08-14 16:38:52,908] {podaac_data_downloader.py:358} INFO - 2024-08-14 16:38:52.908469 SUCCESS: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_141_246L_20240513T084824_20240513T084835_PIC0_01.nc
[2024-08-14 16:39:28,268] {podaac_data_downloader.py:358} INFO - 2024-08-14 16:39:28.268590 SUCCESS: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_141_245R_20240513T084814_20240513T084825_PIC0_01.nc
[2024-08-14 16:40:07,303] {podaac_data_downloader.py:358} INFO - 2024-08-14 16:40:07.303248 SUCCESS: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_064L_20240508T192414_20240508T192425_PIC0_01.nc
[2024-08-14 16:40:34,427] {podaac_data_downloader.py:358} INFO - 2024-08-14 16:40:34.427236 SUCCESS: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_063L_20240508T192404_20240508T192415_PIC0_01.nc
[2024-08-14 16:41:01,101] {podaac_data_downloader.py:358} INFO - 2024-08-14 16:41:01.101385 SUCCESS: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_062L_20240508T192354_20240508T192405_PIC0_01.nc
[2024-08-14 16:41:01,102] {podaac_data_downloader.py:369} INFO - Downloaded Files: 6
[2024-08-14 16:41:01,102] {podaac_data_downloader.py:370} INFO - Failed Files: 0
[2024-08-14 16:41:01,102] {podaac_data_downloader.py:371} INFO - Skipped Files: 0
[2024-08-14 16:41:01,102] {podaac_data_downloader.py:177} INFO - Success Count: 6
[2024-08-14 16:41:01,347] {podaac_data_downloader.py:186} INFO - END

trevor.wilkerson
Posts: 5
Joined: Wed Aug 14, 2024 2:51 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by trevor.wilkerson » Mon Aug 19, 2024 2:26 pm America/New_York

Hello,

I have tried it again, and once again it fails. Sounds like it might be something on my end...
 
(interactive_tool) (base) C:\interactive_tool>podaac-data-downloader -c SWOT_L2_HR_PIXC_2.0 -d tempfolder --start-date 2024-05-04T19:48:32Z --end-date 2024-05-14T19:48:43Z -b=-0.13698,51.48410,-0.04806,52.50708
[2024-08-19 12:13:05,750] {podaac_data_downloader.py:317} INFO - Found 6 total files to download
[2024-08-19 12:13:06,280] {podaac_access.py:503} WARNING - Computed checksum 3a82a9f62a04b2acfa1afce75eb61049 does not match expected checksum 0c5fec1e89b05697c711c8dbbcfd5813


Here is the full output for reference (maybe there is something usefull):

(interactive_tool) (base) C:\interactive_tool>podaac-data-downloader -c SWOT_L2_HR_PIXC_2.0 -d tempfolder --start-date 2024-05-04T19:48:32Z --end-date 2024-05-14T19:48:43Z -b=-0.13698,51.48410,-0.04806,52.50708
[2024-08-19 12:13:05,750] {podaac_data_downloader.py:317} INFO - Found 6 total files to download
[2024-08-19 12:13:06,280] {podaac_access.py:503} WARNING - Computed checksum 3a82a9f62a04b2acfa1afce75eb61049 does not match expected checksum 0c5fec1e89b05697c711c8dbbcfd5813
[2024-08-19 12:20:31,326] {podaac_data_downloader.py:366} WARNING - 2024-08-19 12:20:31.326910 FAILURE: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_141_247L_20240513T084834_20240513T084845_PIC0_01.nc
Traceback (most recent call last):
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_data_downloader.py", line 354, in cmr_downloader
pa.download_file(f,output_path)
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_access.py", line 352, in download_file
urlretrieve(remote_file, output_path)
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 280, in urlretrieve
raise ContentTooShortError(
urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 24245162 out of 566237037 bytes>
[2024-08-19 12:20:31,584] {podaac_data_downloader.py:366} WARNING - 2024-08-19 12:20:31.584333 FAILURE: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_141_246L_20240513T084824_20240513T084835_PIC0_01.nc
Traceback (most recent call last):
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1286, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1041, in _send_output
self.send(msg)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 979, in send
self.connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1451, in connect
super().connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 945, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 827, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_data_downloader.py", line 354, in cmr_downloader
pa.download_file(f,output_path)
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_access.py", line 352, in download_file
urlretrieve(remote_file, output_path)
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
[2024-08-19 12:20:31,781] {podaac_data_downloader.py:366} WARNING - 2024-08-19 12:20:31.780339 FAILURE: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_141_245R_20240513T084814_20240513T084825_PIC0_01.nc
Traceback (most recent call last):
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1286, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1041, in _send_output
self.send(msg)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 979, in send
self.connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1451, in connect
super().connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 945, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 827, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_data_downloader.py", line 354, in cmr_downloader
pa.download_file(f,output_path)
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_access.py", line 352, in download_file
urlretrieve(remote_file, output_path)
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
[2024-08-19 12:20:31,979] {podaac_data_downloader.py:366} WARNING - 2024-08-19 12:20:31.979938 FAILURE: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_064L_20240508T192414_20240508T192425_PIC0_01.nc
Traceback (most recent call last):
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1286, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1041, in _send_output
self.send(msg)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 979, in send
self.connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1451, in connect
super().connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 945, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 827, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_data_downloader.py", line 354, in cmr_downloader
pa.download_file(f,output_path)
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_access.py", line 352, in download_file
urlretrieve(remote_file, output_path)
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
[2024-08-19 12:20:32,178] {podaac_data_downloader.py:366} WARNING - 2024-08-19 12:20:32.178198 FAILURE: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_063L_20240508T192404_20240508T192415_PIC0_01.nc
Traceback (most recent call last):
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1286, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1041, in _send_output
self.send(msg)
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 979, in send
self.connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 1451, in connect
super().connect()
File "C:\ProgramData\miniconda3\Lib\http\client.py", line 945, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 827, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_data_downloader.py", line 354, in cmr_downloader
pa.download_file(f,output_path)
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_access.py", line 352, in download_file
urlretrieve(remote_file, output_path)
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
[2024-08-19 12:21:18,847] {podaac_data_downloader.py:366} WARNING - 2024-08-19 12:21:18.847868 FAILURE: https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_062L_20240508T192354_20240508T192405_PIC0_01.nc
Traceback (most recent call last):
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_data_downloader.py", line 354, in cmr_downloader
pa.download_file(f,output_path)
File "C:\interactive_tool\interactive_tool\Lib\site-packages\subscriber\podaac_access.py", line 352, in download_file
urlretrieve(remote_file, output_path)
File "C:\ProgramData\miniconda3\Lib\urllib\request.py", line 280, in urlretrieve
raise ContentTooShortError(
urllib.error.ContentTooShortError: <urlopen error retrieval incomplete: got only 78476147 out of 595708055 bytes>
[2024-08-19 12:21:18,849] {podaac_data_downloader.py:369} INFO - Downloaded Files: 0
[2024-08-19 12:21:18,850] {podaac_data_downloader.py:370} INFO - Failed Files: 6
[2024-08-19 12:21:18,851] {podaac_data_downloader.py:371} INFO - Skipped Files: 0
[2024-08-19 12:21:18,852] {podaac_data_downloader.py:177} INFO - Success Count: 0
[2024-08-19 12:21:18,853] {podaac_data_downloader.py:186} INFO - END

PODAAC - celiaoued
Subject Matter Expert
Subject Matter Expert
Posts: 46
Joined: Fri May 28, 2021 1:30 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by PODAAC - celiaoued » Tue Aug 20, 2024 7:43 pm America/New_York

What version of podaac-data-subscriber are you running?
How about clearing your tempfolder and trying the command again?

trevor.wilkerson
Posts: 5
Joined: Wed Aug 14, 2024 2:51 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by trevor.wilkerson » Wed Aug 21, 2024 1:49 pm America/New_York

I am using podaac-data-subscriber 1.15.2
on Python 3.11.5

I have tried a fresh venv with only the podaac-data-subscriber installled. I have used an anaconda environment, a python environment. I have tried running it on my CMD and on a miniconda prompt. Same issue every time. It always starts the download, then fails. It works fine on other data products (such as the raster) that are typically smaller.

I have run (basic) network stability tests that seem to show no issue.
 
I have tried deleting the tempfolder and its contents and rerunning it to no avail. It seems like it might be how the checksum is being computed on my end? I am not sure exactly how to troubleshoot this issue.

Next, I will try running this on a different machine. Will keep the thread updated.

Any advice is appreciated.

PODAAC - celiaoued
Subject Matter Expert
Subject Matter Expert
Posts: 46
Joined: Fri May 28, 2021 1:30 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by PODAAC - celiaoued » Wed Aug 21, 2024 3:07 pm America/New_York

We still suspect an unstable network issue. Are you able to access another network to download from?
Also, could you try downloading one of the files manually? Like https://archive.swot.podaac.earthdata.nasa.gov/podaac-swot-ops-cumulus-protected/SWOT_L2_HR_PIXC_2.0/SWOT_L2_HR_PIXC_015_014_062L_20240508T192354_20240508T192405_PIC0_01.nc

trevor.wilkerson
Posts: 5
Joined: Wed Aug 14, 2024 2:51 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by trevor.wilkerson » Wed Aug 21, 2024 5:31 pm America/New_York

No issue downloading through a browser, took about 20 seconds.

PODAAC - celiaoued
Subject Matter Expert
Subject Matter Expert
Posts: 46
Joined: Fri May 28, 2021 1:30 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by PODAAC - celiaoued » Wed Aug 21, 2024 7:18 pm America/New_York

Then, can I ask you to try the command again and capture the output? And if it still fails, can you use a different network?

trevor.wilkerson
Posts: 5
Joined: Wed Aug 14, 2024 2:51 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by trevor.wilkerson » Thu Aug 29, 2024 4:02 pm America/New_York

So I tried it on a different network and continued to get the checksum error:

WARNING - Computed checksum 1bf9ab5f306f61485d2c2e2b56bcda1c does not match expected checksum 0c5fec1e89b05697c711c8dbbcfd5813

But downloaded all 6 with success.

I was on a secure network before and there must have been something interfering with the download via podaac-data-subscriber. No idea what. But issue resolved. It was related to network instability in some way.

Thanks for all the advice.

PODAAC - celiaoued
Subject Matter Expert
Subject Matter Expert
Posts: 46
Joined: Fri May 28, 2021 1:30 pm America/New_York
Answers: 0

Re: SWOT_L2_HR_PIXC_2.0 scripting download

by PODAAC - celiaoued » Thu Aug 29, 2024 6:30 pm America/New_York

Thank you for continuing to try it while reporting the errors and your feedback!

Post Reply