Errors accessing HLS data

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
venusinkant
Posts: 2
Joined: Thu Apr 04, 2024 4:05 am America/New_York
Answers: 0
Been thanked: 1 time

Re: Errors accessing HLS data

by venusinkant » Sun Apr 07, 2024 10:54 pm America/New_York

I have a same error. I am trying to access HLS data according to the https://github.com/nasa/HLS-Data-Resources/blob/main/python/tutorials/HLS_Tutorial.ipynb. But I get the error
vsi curl error: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T22KDB.2018308T133219.v2.0/HLS.S30.T22KDB.2018308T133219.v2.0.B04.tif' not recognized as a supported file format.. Retrying...

Could you tell me how do you deal with this problem?

Thank you

Filters:

LP DAAC - dgolon
User Services
User Services
Posts: 338
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 24 times
Been thanked: 4 times
Contact:

Re: Errors accessing HLS data

by LP DAAC - dgolon » Thu Apr 11, 2024 9:37 am America/New_York

Hi @venusinkant Our science team is looking into this. We'll report here when we have more info. Thanks --Danielle
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

LP DAAC - dgolon
User Services
User Services
Posts: 338
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 24 times
Been thanked: 4 times
Contact:

Re: Errors accessing HLS data

by LP DAAC - dgolon » Thu Apr 11, 2024 3:59 pm America/New_York

Our science team continues to look into the error you are seeing but are having trouble replicating it. In the meantime, they recommend using AppEEARS or the AppEEARS API to access the HLS data: https://appeears.earthdatacloud.nasa.gov/
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

venusinkant
Posts: 2
Joined: Thu Apr 04, 2024 4:05 am America/New_York
Answers: 0
Been thanked: 1 time

Re: Errors accessing HLS data

by venusinkant » Fri Apr 12, 2024 3:00 am America/New_York

Thank you for the reply!

I have found the solutuion to my problem. The python version in my cmd is 3.6, but the environment where I run the code is python 3.11. When I run the code to logging into my earthdata account, the program create the netrc file in my computer by the python3.6, but when I run the following code, the environment is python 3.11. So comes the error.

Thank you again for your help!

kvenkataramani
Posts: 1
Joined: Mon Apr 15, 2024 5:27 pm America/New_York
Answers: 0
Been thanked: 1 time

Re: Errors accessing HLS data

by kvenkataramani » Mon Apr 15, 2024 5:31 pm America/New_York

My '.netrc' file was missing a newline character at the end of the line containing my earthdata credentials. Adding this to my file on my Windows machine fixed the issue.

The gdalinfo command did note that it was unable to open the remote URL, pinpointing the error a credentials issue. Then I tested out changing the read/write permissions for the file as well as adding the newline character at the end of the line. The latter fixed the issue.

rbrassfield
Posts: 1
Joined: Mon Apr 22, 2024 9:35 pm America/New_York
Answers: 0

Re: Errors accessing HLS data

by rbrassfield » Thu Apr 25, 2024 3:13 pm America/New_York

I'm running into the same issues. I'm working within fully updated packages/environments following this GDAL config:
gdal.SetConfigOption('GDAL_HTTP_COOKIEFILE','~/cookies.txt')
gdal.SetConfigOption('GDAL_HTTP_COOKIEJAR', '~/cookies.txt')
gdal.SetConfigOption('GDAL_DISABLE_READDIR_ON_OPEN','EMPTY_DIR')
gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS','tif')
gdal.SetConfigOption('GDAL_HTTP_UNSAFESSL', 'YES')

with the following lines for building/double checking the netrc file:
urs = 'urs.earthdata.nasa.gov' # Earthdata URL endpoint for authentication
prompts = ['Enter NASA Earthdata Login Username: ',
'Enter NASA Earthdata Login Password: ']

netrc_name = ".netrc"

# Determine if netrc file exists, and if so, if it includes NASA Earthdata Login Credentials
try:
netrcDir = os.path.expanduser(f"~/{netrc_name}")
netrc(netrcDir).authenticators(urs)[0]

# Below, create a netrc file and prompt user for NASA Earthdata Login Username and Password
except FileNotFoundError:
homeDir = os.path.expanduser("~")
Popen('touch {0}{2} | echo machine {1} >> {0}{2}'.format(homeDir + os.sep, urs, netrc_name), shell=True)
Popen('echo login {} >> {}{}'.format(getpass(prompt=prompts[0]), homeDir + os.sep, netrc_name), shell=True)
Popen('echo \'password {} \'>> {}{}'.format(getpass(prompt=prompts[1]), homeDir + os.sep, netrc_name), shell=True)
# Set restrictive permissions
Popen('chmod 0600 {0}{1}'.format(homeDir + os.sep, netrc_name), shell=True)

# Determine OS and edit netrc file if it exists but is not set up for NASA Earthdata Login
except TypeError:
homeDir = os.path.expanduser("~")
Popen('echo machine {1} >> {0}{2}'.format(homeDir + os.sep, urs, netrc_name), shell=True)
Popen('echo login {} >> {}{}'.format(getpass(prompt=prompts[0]), homeDir + os.sep, netrc_name), shell=True)
Popen('echo \'password {} \'>> {}{}'.format(getpass(prompt=prompts[1]), homeDir + os.sep, netrc_name), shell=True)

And I am still getting the following error:
---------------------------------------------------------------------------
CPLE_OpenFailedError Traceback (most recent call last)
rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()

rasterio/_base.pyx in rasterio._base.open_dataset()

rasterio/_err.pyx in rasterio._err.exc_wrap_pointer()

CPLE_OpenFailedError: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif' not recognized as a supported file format.

During handling of the above exception, another exception occurred:

RasterioIOError Traceback (most recent call last)
<ipython-input-29-7e3d3a89d91f> in <cell line: 1>()
----> 1 rio.open("https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif")

1 frames
/usr/local/lib/python3.10/dist-packages/rasterio/__init__.py in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, **kwargs)
302
303 if mode == "r":
--> 304 dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
305 elif mode == "r+":
306 dataset = get_writer_for_path(path, driver=driver)(

rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()

RasterioIOError: '/vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T11TMK.2016036T190112.v2.0/HLS.S30.T11TMK.2016036T190112.v2.0.B02.tif' not recognized as a supported file format.

I have too much data for the AppEEARS download methods.

LP DAAC - dgolon
User Services
User Services
Posts: 338
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 24 times
Been thanked: 4 times
Contact:

Re: Errors accessing HLS data

by LP DAAC - dgolon » Fri Apr 26, 2024 1:41 pm America/New_York

Hi @rbrassfield We have worked with a few users now that previously found this error and it seems resetting their Earthdata Login password (https://urs.earthdata.nasa.gov/reset_passwords/new) has resolved the issue. When creating a password please avoid using the "#", "&" or "%" symbols or other similar characters in it, as in some cases that also seems to have caused an issue. If resetting your password and then trying again does not work, please email us a copy of the exact script you are using to lpdaac@usgs.gov and reference this post. Thanks --Danielle
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

Post Reply