Page 1 of 1

Direct access to S3 data from outside EC2

Posted: Thu Jun 01, 2023 9:18 am America/New_York
by lesimppa
Hi everyone,

Thanks for making the data public. It's awesome and very easy to download with the HTTP endpoint. We are building a distributed system that consumes data from LPDAAC and the HTTP endpoint is a little slow for our needs. I've been trying to access assets via S3 as shown in this notebook: https://nasa-openscapes.github.io/2021-Cloud-Hackathon/tutorials/05_Data_Access_Direct_S3.html, which seems not be working as I'm doing it from outside AWS. I understand that direct access must be done from within the same region, but would there be a way do it from outside, even if it's not free?

Thank you,
Simon

Re: Direct access to S3 data from outside EC2

Posted: Fri Jun 02, 2023 3:22 pm America/New_York
by LP DAAC - afriesz
@lesimppa ,

The notebook you refer to will only work if you are executing the notebook in AWS region us-west-2. If you are trying to access data that resides in Earthdata Cloud from a machine outside AWS, e.g. from your laptop, you have to use the HTTPS endpoint. This notebook gives a quick Python example for setting the environmental variables and reading in an HLS asset via its HTTPS URL: https://github.com/NASA-Openscapes/2021-Cloud-Workshop-AGU/blob/main/how-tos/Earthdata_Cloud__Single_File__HTTPS_Access_COG_Example.ipynb.