ASDC Response: 401/Unauthorized when using python3
Posted: Thu Mar 17, 2022 7:07 pm America/New_York
I am trying to download CALIPSO data from ASDC using a python. scripts based on the examples on this forum. My example code is very simple
import requests
session = requests.Session()
session.auth = (<my username>, <my password>)
url = 'https://asdc.larc.nasa.gov/data/CALIPSO/LID_L1-Standard-V4-10/2007/01/CAL_LID_L1-Standard-V4-10.2007-01-01T00-22-49ZN.hdf'
r1 = session.get(url)
This returns a 401 response (unauthorised). I have my username and password correct, as I am able to download a MODIS file using the same code
url = 'https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD08_D3/2020/001/MYD08_D3.A2020001.061.2020003014337.hdf'
r1 = session.request('get', url)
Is a status: 200 (success). I can download this file using the wget instructions, so this appears to be a problem with the python code, but given this is a basic example, I am not sure what the issue is. The library debug output is below (with some bit removed)
Many thanks
Ed
----
In[9]: session.get(url)
DEBUG:urllib3.connectionpool:Resetting dropped connection: asdc.larc.nasa.gov
send: b'GET /data/CALIPSO/LID_L1-Standard-V4-10/2007/01/CAL_LID_L1-Standard-V4-10.2007-01-01T00-22-49ZN.hdf HTTP/1.1\r\nHost: asdc.larc.nasa.gov\r\nUser-Agent: python-requests/2.27.1\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nCookie: ----------------\r\nAuthorization: Basic -------------------\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Thu, 17 Mar 2022 22:55:32 GMT
header: Server: Apache/2.4.34 (Red Hat) OpenSSL/1.0.2k-fips
header: Content-Length: 381
header: Content-Type: text/html; charset=iso-8859-1
header: Strict-Transport-Security: max-age=31536000;includeSubDomains;preload
DEBUG:urllib3.connectionpool:https://asdc.larc.nasa.gov:443 "GET /data/CALIPSO/LID_L1-Standard-V4-10/2007/01/CAL_LID_L1-Standard-V4-10.2007-01-01T00-22-49ZN.hdf HTTP/1.1" 401 381
Out[9]: <Response [401]>
import requests
session = requests.Session()
session.auth = (<my username>, <my password>)
url = 'https://asdc.larc.nasa.gov/data/CALIPSO/LID_L1-Standard-V4-10/2007/01/CAL_LID_L1-Standard-V4-10.2007-01-01T00-22-49ZN.hdf'
r1 = session.get(url)
This returns a 401 response (unauthorised). I have my username and password correct, as I am able to download a MODIS file using the same code
url = 'https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MYD08_D3/2020/001/MYD08_D3.A2020001.061.2020003014337.hdf'
r1 = session.request('get', url)
Is a status: 200 (success). I can download this file using the wget instructions, so this appears to be a problem with the python code, but given this is a basic example, I am not sure what the issue is. The library debug output is below (with some bit removed)
Many thanks
Ed
----
In[9]: session.get(url)
DEBUG:urllib3.connectionpool:Resetting dropped connection: asdc.larc.nasa.gov
send: b'GET /data/CALIPSO/LID_L1-Standard-V4-10/2007/01/CAL_LID_L1-Standard-V4-10.2007-01-01T00-22-49ZN.hdf HTTP/1.1\r\nHost: asdc.larc.nasa.gov\r\nUser-Agent: python-requests/2.27.1\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nCookie: ----------------\r\nAuthorization: Basic -------------------\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Thu, 17 Mar 2022 22:55:32 GMT
header: Server: Apache/2.4.34 (Red Hat) OpenSSL/1.0.2k-fips
header: Content-Length: 381
header: Content-Type: text/html; charset=iso-8859-1
header: Strict-Transport-Security: max-age=31536000;includeSubDomains;preload
DEBUG:urllib3.connectionpool:https://asdc.larc.nasa.gov:443 "GET /data/CALIPSO/LID_L1-Standard-V4-10/2007/01/CAL_LID_L1-Standard-V4-10.2007-01-01T00-22-49ZN.hdf HTTP/1.1" 401 381
Out[9]: <Response [401]>