Earthdata Forum

Hello,

I have a set of code that builds an HLS data cube over a given location and time-interval, then runs further processing. As part of that, this data cube needs to be loaded into memory with persist().

The code works well in most scenarios. However, I have found that one particular tif is unable to be accessed, triggering an error and preventing the HLS data cube from being loaded into memory.

The relevant portions of the error:

CPLE_OpenFailedError: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10UDU.2014082T190147.v2.0/HLS.L30.T10UDU.2014082T190147.v2.0.B06.tif' does not exist in the file system, and is not recognized as a supported dataset name.

RuntimeError: Error opening 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10UDU.2014082T190147.v2.0/HLS.L30.T10UDU.2014082T190147.v2.0.B06.tif': RasterioIOError("'/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10UDU.2014082T190147.v2.0/HLS.L30.T10UDU.2014082T190147.v2.0.B06.tif' does not exist in the file system, and is not recognized as a supported dataset name.")

Note that it is always the same tif link that is provided by the error, and retrying the function triggers the same error.

Any help would be appreciated!

Hi @mitchbon I've passed your question along to our developers. We will reach back on on this post once we have an answer or if we need additional information from you. Thanks!

I ran the same code again today and it was able to successfully find and load that problematic tif into memory. I guess that means this is some sort of server issue?

@mitchbon

If you are unable to load EarthData assets from https URLs via vsicurl and rasterio in Python, there are 3 common solutions:

1. Ensure you have a properly configured .netrc file. Instructions can be found [here|https://github.com/nasa/LPDAAC-Data-Resources/blob/main/notebooks/Earthdata_Authentication__Create_netrc_file.ipynb].

2.Ensure that you have set the necessary gdal configurations to access data using vsicurl. The below code can be used to set these:
gdal.SetConfigOption('GDAL_HTTP_COOKIEFILE','~/cookies.txt')
gdal.SetConfigOption('GDAL_HTTP_COOKIEJAR', '~/cookies.txt')
gdal.SetConfigOption('GDAL_DISABLE_READDIR_ON_OPEN','EMPTY_DIR')
gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS','TIF')

3. Sometimes cached information can cause an issue, to resolve this, try restarting your Python Kernel.