VIIRS and MODIS Level 2 Subscription Access

[oceani]

I am getting a "ERROR 308: PERMANENT REDIRECT" error when trying to access my VIIRS and MODIS (both Aqua and Terra) SST and CHL Level 2 subscription data using wget and the URL https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/{filename} which appears to be rerouted to https://oceandata.sci.gsfc.nasa.gov/getfile/{filename}. This seems to be a very recent situation and is happening for all of our subscription IDs. A few sample subscription numbers would be: 2625, 2971 and 2559. Has the URL changed or anything else I should know about when trying to access subscription files?

Thanks,

Mark

[OB ODPS - jgwilding]

Hi Mark.

We have been developing a new version of the getfile application, and the redirect is in place to deal with the old end points. Is the redirect creating a problem for you or is this just an observation?


john

[oceani]

Hi John,

Thanks for the quick reply. Yes. As of 2:15 PM MDT the redirect is still ending up in "ERROR 308: PERMANENT REDIRECT".

-Mark

[OB WebDev - masoudso]

Hi Mark,
That warning is showing that the old urls are now pointing to the new url. May I know if it is causing a problem when you try to download the data?

Thanks,
Masoud

[oceani]

Masoud,

The download fails. I see the error message and then the file transfer aborts and moves on to the next file in my script. An example for a file from our subscription for level 2 Aqua SST data ID# 2926 shows:

AQUA_MODIS.20230928T152501.L2.SST.NRT.nc
--2023-09-28 14:45:42-- https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc
Resolving oceandata.sci.gsfc.nasa.gov... 169.154.128.84
Connecting to oceandata.sci.gsfc.nasa.gov|169.154.128.84|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://oceandata.sci.gsfc.nasa.gov/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc [following]
--2023-09-28 14:45:43-- https://oceandata.sci.gsfc.nasa.gov/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc
Connecting to oceandata.sci.gsfc.nasa.gov|169.154.128.84|:443... connected.
HTTP request sent, awaiting response... 308 PERMANENT REDIRECT
2023-09-28 14:45:43 ERROR 308: PERMANENT REDIRECT.

After this, the script either moves on to the next file (and fails) or finishes unsuccessfully.

Mark

[OB ODPS - jgwilding]

Mark,

Are you using wget or curl to download? Can you post the command and its version?

john

[oceani]

John,

wget Version - "GNU Wget 1.12 built on linux-gnu"

Here is an example line from a CShell script that does the retrieval:

wget --user=oceani --password=XXXXXXXX --auth-no-challenge=on --no-check-certificate https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc

Mark

[OB ODPS - jgwilding]

Ok, we're running,

% wget --version
GNU Wget 1.20.3 built on linux-gnu.

The man page mentions a --max-redirect option and claims its default is 20. Maybe your version has a different value for a default. Can you try adding --max-redirect=10 to your command?

john

[oceani]

John,

Same issue. Here is a capture of the command and result:

wget --user=oceani --password=Oceani123! --auth-no-challenge=on --no-check-certificate --max-redirect=10 https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/SNPP_VIIRS.20230928T035400.L2.SST.NRT.nc
--2023-09-28 16:14:37-- https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/SNPP_VIIRS.20230928T035400.L2.SST.NRT.nc
Resolving oceandata.sci.gsfc.nasa.gov... 169.154.128.84
Connecting to oceandata.sci.gsfc.nasa.gov|169.154.128.84|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://oceandata.sci.gsfc.nasa.gov/getfile/SNPP_VIIRS.20230928T035400.L2.SST.NRT.nc [following]
--2023-09-28 16:14:37-- https://oceandata.sci.gsfc.nasa.gov/getfile/SNPP_VIIRS.20230928T035400.L2.SST.NRT.nc
Connecting to oceandata.sci.gsfc.nasa.gov|169.154.128.84|:443... connected.
HTTP request sent, awaiting response... 308 PERMANENT REDIRECT
2023-09-28 16:14:38 ERROR 308: PERMANENT REDIRECT.

Just goes back to the command line after that.

-Mark

[OB ODPS - jgwilding]

That seems odd. The 308 status is not meant to be a fatal error. Older versions of wget had a bug that prevented them from following the redirects. Any way to upgrade or switch to curl?

https://lists.gnu.org/archive/html/bug-wget/2017-11/msg00015.html

Code: Select all

% wget --max-redirect=2 --no-check-certificate -O /dev/null  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc
--2023-09-28 22:24:10--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... fddd:0:0:2::56, 192.168.2.56
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|fddd:0:0:2::56|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://oceandata.sci.gsfc.nasa.gov/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc [following]
--2023-09-28 22:24:10--  https://oceandata.sci.gsfc.nasa.gov/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc
Reusing existing connection to [oceandata.sci.gsfc.nasa.gov]:443.
HTTP request sent, awaiting response... 308 PERMANENT REDIRECT
Location: https://oceandata.sci.gsfc.nasa.gov/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc/ [following]
--2023-09-28 22:24:11--  https://oceandata.sci.gsfc.nasa.gov/getfile/AQUA_MODIS.20230928T152501.L2.SST.NRT.nc/
Reusing existing connection to [oceandata.sci.gsfc.nasa.gov]:443.
HTTP request sent, awaiting response... 200 OK
Length: 19967451 (19M) [application/octet-stream]
Saving to: '/dev/null'

/dev/null                          100%[=============================================================>]  19.04M  10.0MB/s    in 1.9s    

2023-09-28 22:24:15 (10.0 MB/s) - '/dev/null' saved [19967451/19967451]