Possible rate limiting or throttling on LP DAAC HTTPS downloads when accessing HLS COGs in parallel

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
thaleslc
Posts: 1
Joined: Sat Aug 16, 2025 6:27 am America/New_York
Answers: 0

Possible rate limiting or throttling on LP DAAC HTTPS downloads when accessing HLS COGs in parallel

by thaleslc » Wed Dec 24, 2025 2:09 am America/New_York

I am retrieving HLS data links using the earthaccess Python library. I am able to access subsets of the desired COGs directly via S3, and this workflow is both fast and highly parallelizable (I am using rasterio + dask).

However, when I attempt to parallelize the same workflow using the HTTPS links (i.e., not direct S3 access), I observe a different behavior. Initially, the requests run somewhat slower than S3 access, which is expected since I am accessing the data from outside the AWS region. After some time, however, the performance degrades significantly: individual requests that initially take around 2–5 seconds begin to take 100–300 seconds, and eventually some of them fail.

Given this, I would like to ask:

1 - Are there any limits on the number of concurrent or time-based HTTPS download requests to the LP DAAC Earthdata Cloud endpoints?
2 - Is there any form of rate limiting, throttling, or fair-use policy applied to HTTPS file downloads that could explain this progressive slowdown?

I understand that APIs often have documented rate limits, but in this case I am referring specifically to file downloads via HTTPS rather than API calls.
Any clarification or pointers to relevant documentation would be greatly appreciated.

Thank you.

Filters:

LP DAAC - lien
User Services
User Services
Posts: 52
Joined: Mon Mar 17, 2025 2:04 pm America/New_York
Answers: 1
Endorsed: 6 times

Re: Possible rate limiting or throttling on LP DAAC HTTPS downloads when accessing HLS COGs in parallel

by LP DAAC - lien » Wed Dec 31, 2025 2:52 pm America/New_York

Hello,
Sorry for the delays. We are having our developers look into this. The https will download a little slower, but 1-5 minutes is much longer then it should. I will let you know as soon as we hear more.
Thanks,
Brett

Post Reply