I have a python code (the second code from https://urs.earthdata.nasa.gov/documentation/for_users/data_access/python) to configure a connection and download data from Earthdata. In 2022 I could download thousands of URLs using that scripts. But starting January 2023 the code hasn't been working for me and I cannot figure out what is wrong.
Once I execute the code, it runs and downloads empty files - they are just 20KB and do not contain imagery data. If I copy the URL in browser, it donwloads the imagery and the files are ~150MB.
An example URL is: "https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/5200/VNP02MOD/2022/365/VNP02MOD.A2022365.2142.002.2023001063801.nc"
Something is wrong and I can't figure out what it is. Any help would be much appreciated.
They are likely HTML files. If so, open one in a browser and there will likely be a message about an authentication error when using the script.
I know how to generate tokens from NRT, but don't know how to use them in a Python script.
The python examples I've found on earthdata use username/password. Like the ones here:
So far, we know this is an authentication issue that is happening to python codes after January 2023.
I configured the arguments with the URL and a token. But it throws the following error: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte.
This happens at this line 'return fh.read().decode('utf-8')'. I have tried several approaches that I found on the Internet, but nothing seems to work for this particular case.
Any ideas how to decode this correctly?
I am running the code in Python 3 and Windows.
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
Example of the URLs that I am trying to download is:
The file contains NetCDF4 scenes captured by S-NPP VIIRS.
I am using Python 3 in Windows. I am also using Anaconda for environment management and run my codes in Spyder.
curl --fail --header "Authorization: Bearer YOUR_DOWNLOAD_TOKEN" -sS -L -b session --get https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/5200/VNP02MOD/2022/365/VNP02MOD.A2022365.2142.002.2023001063801.nc > VNP02MOD.A2022365.2142.002.2023001063801.nc
or use wget. The LAADSWeb archive pages have examples of how to use wget.