Incorrect File Size for Aqua, Terra and VIIRS SST and Chl files

[OB ODPS - towens]

Put curly braces around the list variable and escape the question mark with a backslash

Code: Select all

echo ${list[1]}\?appkey=xxxxxxxxx
AQUA_MODIS.20231207T203500.L2.SST.NRT.nc?appkey=xxxxxxxxx

Tommy

[pmlrsg]

We are having similar issues here at PML. We are using python requests library to download files. How do we pass the appkey/token to the get request? We are recieving the same 'invalid appkey/token used' when running requests.get('https://oceandata.sci.gsfc.nasa.gov/ob/getfile/JPSS1_VIIRS.20231210T003601.L2.OC.NRT.nc')

Thanks

[OB ODPS - towens]

Does it not work if append it to the end of the URL?:
requests.get('https://oceandata.sci.gsfc.nasa.gov/ob/getfile/JPSS1_VIIRS.20231210T003601.L2.OC.NRT.nc?appkey=xxxxxxx')

Tommy

[pmlrsg]

Yes it does, I was wondering if there was a slightly better/more secure method for this. ie. we were using .netrc authentication prior to this.

[OB.DAAC - SeanBailey]

The obdaac_download script (available via https://oceancolor.gsfc.nasa.gov/data/download_methods/) is a python script that uses the .netrc file for authentication. It also can use the appkey approach. I've just reconfirmed that both methods work. How are you setting up your requests call (it should work with a .netrc file in place...)?

Sean

[oceani]

Is the appkey approach the new recommended method if using wget to download, or will you be reverting back to how the system functioned prior to Dec 6th?

Thanks,

Mark

[gbourdin]

Hi all,
I have been having exactly the same issue downloading using requests in python. I also tested wget and had the same problem, but no issue when passing the url into a web browser. I was trying to adapt my python work flow to use obdaac_download but it might be faster to fix how I set my request instead.
So far my code was like this:

Code: Select all

with requests.Session() as s:
    r = s.get('https://oceandata.sci.gsfc.nasa.gov/ob/getfile/A2023227115000.L1A_LAC.bz2', auth=(username, password), stream=True, timeout=900)
    with open('tmp_A2023227115000.L1A_LAC.bz2', "ab") as handle:
        for chunk in r.iter_content(chunk_size=16 * 1024):
            if chunk:
                handle.write(chunk)

But it looks like the username/password method doesn't work anymore. I could use some help to set the request to work again.
Also I recently had issues with incomplete files downloaded, I wanted to add a check to ensure the entire file is downloaded, but I didn't find the size of the file in the requests response.
Best,
Guillaume

[OB.DAAC - SeanBailey]

Oh what a tangled web...

We have identified and corrected the issue that was causing the basic authentication of username/password to fail. All *should* be working again...

Please let us know if you're still having issues.

Sean

[gbourdin]

It works perfectly for me thank you so much for your reactivity.
Best,
Guillaume

[oceani]

Thank you for taking care of that. Would you recommend one method over the other (i.e. appkey vs. using user name and password)?

-Mark