obdaac_download.py hangs with only partial download

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
haag
Posts: 33
Joined: Wed Jun 08, 2016 7:06 am America/New_York
Answers: 0

obdaac_download.py hangs with only partial download

by haag » Wed Mar 01, 2023 2:27 pm America/New_York

Hello,

I am seeing the tried-and-true obdaac_download.py script routinely hang, with only occasional success, but the GET request seems to succeed, as the run below, with extra verbosity, shows:

Code: Select all

$ obdaac_download.py -vvv  T2023059155000.L1A_LAC.bz2
Output directory: <elided>
Retrieving T2023059155000.L1A_LAC.bz2
Session started
DEBUG:urllib3.util.retry:Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): oceandata.sci.gsfc.nasa.gov:443
DEBUG:urllib3.connectionpool:https://oceandata.sci.gsfc.nasa.gov:443 "GET /ob/getfile/T2023059155000.L1A_LAC.bz2 HTTP/1.1" 302 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): urs.earthdata.nasa.gov:443
DEBUG:urllib3.connectionpool:https://urs.earthdata.nasa.gov:443 "GET /oauth/authorize?response_type=code&client_id=Z0u-MdLNypXBjiDREZ3roA&redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict HTTP/1.1" 302 None
DEBUG:urllib3.connectionpool:https://oceandata.sci.gsfc.nasa.gov:443 "GET /ob/getfile/restrict?code=0c4fcca48838232823267edbae6e5ce7fbe3978c680432af15463a6e3cf92c25 HTTP/1.1" 302 None
DEBUG:urllib3.connectionpool:https://oceandata.sci.gsfc.nasa.gov:443 "GET /ob/getfile/T2023059155000.L1A_LAC.bz2 HTTP/1.1" 200 206870954
And the above just hangs until I abort. Sometimes, deleting the partial download and retrying will succeed, but mostly not.

Is there a known network issue? My IT people say all is good at my end.

Many thanks!
Alaric

Filters:

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1474
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

Re: obdaac_download.py hangs with only partial download

by OB.DAAC - SeanBailey » Fri Mar 17, 2023 8:04 am America/New_York

Alaric,
Seems we missed this post. Sorry about the long delay in replying. Are you still experiencing the issue? I was not able to replicate it, but that doesn't mean that the problem went away for you.

Sean

haag
Posts: 33
Joined: Wed Jun 08, 2016 7:06 am America/New_York
Answers: 0

Re: obdaac_download.py hangs with only partial download

by haag » Sun Mar 19, 2023 8:04 pm America/New_York

Sean,

No worries; and thank you! I got confirmation from your end that I wasn’t being throttled, but it persisted for days. I managed to ultimately confirm it was likely a routing problem. Consistently, testing the download script over on my laptop, I kept getting a few successful downloads, then the hangs when connected to my campus wired net, but NOT on Wi-Fi. I had a meet-up with IT support here scheduled for the next day, but, as mysteriously as the problem arose, it had vanished by that morning, before we could diagnose the routing.

I have to presume someone somewhere between here and there finally kicked a misbehaving router…

Alaric

Post Reply