Page 1 of 1

Bulk downloading encounters connection errors

Posted: Fri Aug 19, 2022 3:52 am America/New_York
by vds_downloader
I am bulk downloading some GPM SSMI data from GES DISC with wget and am experiencing connection timeouts:

Connecting to gpm1.gesdisc.eosdis.nasa.gov (gpm1.gesdisc.eosdis.nasa.gov)|198.118.197.50|:443... failed: Connection timed out.
Connecting to gpm1.gesdisc.eosdis.nasa.gov (gpm1.gesdisc.eosdis.nasa.gov)|2001:4d0:241a:4041::50|:443... failed: Network is unreachable.

Is this because of a limit on the number of sequential (or concurrent) requests in a certain period of time? And if so, can this limit be increased?

Re: Bulk downloading encounters connection errors

Posted: Fri Aug 19, 2022 10:55 am America/New_York
by GES DISC - mgreene
Thank you for your interest in using GPM data.
In regards to the error you received, the system may issue service temporarily
unavailable for more than one concurrent connection in certain circumstances.
Please review/reduce the number of concurrent connections you are submitting,
and try again.

Re: Bulk downloading encounters connection errors

Posted: Mon Aug 22, 2022 7:23 am America/New_York
by vds_downloader
Thank you for you reaction. Indeed when I decrease the number of wget processes running in parallel I get much less Connection timeouts. But when I have only one wget running I still get the Connection timeouts. It runs for 30 seconds or so and downloads a couple of files but then stalls more than a minute.

Re: Bulk downloading encounters connection errors

Posted: Tue Aug 23, 2022 1:21 pm America/New_York
by GES DISC - mgreene
Our staff are working on this issue. Once I hear back from them I will be in contact with you.
Thanks