I'm running into the same issues. I'm working within fully updated packages/environments following this GDAL config:
gdal.SetConfigOption('GDAL_HTTP_COOKIEFILE','~/cookies.txt')
gdal.SetConfigOption('GDAL_HTTP_COOKIEJAR', '~/cookies.txt')
gdal.SetConfigOption('GDAL_DISABLE_READDIR_ON_OPEN','EMPTY_DIR')
gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS','tif')
gdal.SetConfigOption('GDAL_HTTP_UNSAFESSL', 'YES')
with the following lines for building/double checking the netrc file:
urs = 'urs.earthdata.nasa.gov' # Earthdata URL endpoint for authentication
prompts = ['Enter NASA Earthdata Login Username: ',
'Enter NASA Earthdata Login Password: ']
netrc_name = ".netrc"
# Determine if netrc file exists, and if so, if it includes NASA Earthdata Login Credentials
try:
netrcDir = os.path.expanduser(f"~/{netrc_name}")
netrc(netrcDir).authenticators(urs)[0]
# Below, create a netrc file and prompt user for NASA Earthdata Login Username and Password
except FileNotFoundError:
homeDir = os.path.expanduser("~")
Popen('touch {0}{2} | echo machine {1} >> {0}{2}'.format(homeDir + os.sep, urs, netrc_name), shell=True)
Popen('echo login {} >> {}{}'.format(getpass(prompt=prompts[0]), homeDir + os.sep, netrc_name), shell=True)
Popen('echo \'password {} \'>> {}{}'.format(getpass(prompt=prompts[1]), homeDir + os.sep, netrc_name), shell=True)
# Set restrictive permissions
Popen('chmod 0600 {0}{1}'.format(homeDir + os.sep, netrc_name), shell=True)
# Determine OS and edit netrc file if it exists but is not set up for NASA Earthdata Login
except TypeError:
homeDir = os.path.expanduser("~")
Popen('echo machine {1} >> {0}{2}'.format(homeDir + os.sep, urs, netrc_name), shell=True)
Popen('echo login {} >> {}{}'.format(getpass(prompt=prompts[0]), homeDir + os.sep, netrc_name), shell=True)
Popen('echo \'password {} \'>> {}{}'.format(getpass(prompt=prompts[1]), homeDir + os.sep, netrc_name), shell=True)
And I am still getting the following error:
---------------------------------------------------------------------------
CPLE_OpenFailedError Traceback (most recent call last)
rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()
rasterio/_base.pyx in rasterio._base.open_dataset()
rasterio/_err.pyx in rasterio._err.exc_wrap_pointer()
CPLE_OpenFailedError: '/vsicurl/
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif' not recognized as a supported file format.
During handling of the above exception, another exception occurred:
RasterioIOError Traceback (most recent call last)
<ipython-input-29-7e3d3a89d91f> in <cell line: 1>()
----> 1 rio.open("
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif")
1 frames
/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, **kwargs)
302
303 if mode == "r":
--> 304 dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
305 elif mode == "r+":
306 dataset = get_writer_for_path(path, driver=driver)(
rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()
RasterioIOError: '/vsicurl/
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif' not recognized as a supported file format.
I have too much data for the AppEEARS download methods.