Possible rate limiting or throttling on LP DAAC HTTPS downloads when accessing HLS COGs in parallel
Posted: Wed Dec 24, 2025 2:09 am America/New_York
I am retrieving HLS data links using the earthaccess Python library. I am able to access subsets of the desired COGs directly via S3, and this workflow is both fast and highly parallelizable (I am using rasterio + dask).
However, when I attempt to parallelize the same workflow using the HTTPS links (i.e., not direct S3 access), I observe a different behavior. Initially, the requests run somewhat slower than S3 access, which is expected since I am accessing the data from outside the AWS region. After some time, however, the performance degrades significantly: individual requests that initially take around 2–5 seconds begin to take 100–300 seconds, and eventually some of them fail.
Given this, I would like to ask:
1 - Are there any limits on the number of concurrent or time-based HTTPS download requests to the LP DAAC Earthdata Cloud endpoints?
2 - Is there any form of rate limiting, throttling, or fair-use policy applied to HTTPS file downloads that could explain this progressive slowdown?
I understand that APIs often have documented rate limits, but in this case I am referring specifically to file downloads via HTTPS rather than API calls.
Any clarification or pointers to relevant documentation would be greatly appreciated.
Thank you.
However, when I attempt to parallelize the same workflow using the HTTPS links (i.e., not direct S3 access), I observe a different behavior. Initially, the requests run somewhat slower than S3 access, which is expected since I am accessing the data from outside the AWS region. After some time, however, the performance degrades significantly: individual requests that initially take around 2–5 seconds begin to take 100–300 seconds, and eventually some of them fail.
Given this, I would like to ask:
1 - Are there any limits on the number of concurrent or time-based HTTPS download requests to the LP DAAC Earthdata Cloud endpoints?
2 - Is there any form of rate limiting, throttling, or fair-use policy applied to HTTPS file downloads that could explain this progressive slowdown?
I understand that APIs often have documented rate limits, but in this case I am referring specifically to file downloads via HTTPS rather than API calls.
Any clarification or pointers to relevant documentation would be greatly appreciated.
Thank you.