Subscriptions Empty

Please enter here to ask a question about any NASA Science related topics!
oceani
Posts: 33
Joined: Wed Aug 23, 2006 12:32 pm America/New_York

Subscriptions Empty

by oceani » Thu Oct 22, 2020 11:31 am America/New_York

All of the queries for my subscriptions are coming up empty as of this morning.  These are mostly subscriptions for Level 2 MODIS (both Terra and Aqua) and VIIRS SST data.  We have several subscriptions, but a few subscription numbers for reference are 906, 2650 and 2642.  The initial command that we use to see the content of our subscription file listing is:
wget --user=XXXXX --password=XXXXXXX --post-data="subID=906&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search
This command is coming up with zero results for most-to-all of our SST subscriptions. 

Also when I click on the File Search button (far right column) on View Active-->My Subscriptions page it seems to be oscillating between there being no entries to being a full list of data files.  For the past 15-20 minutes or so these listings have been completely empty for all of my subscriptions.

Please advise.

Thanks much,

Mark

Tags:

OB ODPS - jgwilding
Subject Matter Expert
Subject Matter Expert
Posts: 110
Joined: Fri Feb 19, 2021 1:09 pm America/New_York

Subscriptions Empty

by OB ODPS - jgwilding » Thu Oct 22, 2020 12:27 pm America/New_York

Mark,

I don't think you need the user/passwd options for the file search.  When i run this, i get files through today (Oct 22).  The 2> redirects the wget msgs to /dev/null.  Does this work for you?

john

$ wget --post-data="subID=906&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search 2> /dev/null
AQUA_MODIS.20201018T200501.L2.SST.NRT.nc
AQUA_MODIS.20201018T213501.L2.SST.NRT.nc
AQUA_MODIS.20201018T214001.L2.SST.NRT.nc
AQUA_MODIS.20201018T214501.L2.SST.NRT.nc
AQUA_MODIS.20201018T231001.L2.SST.NRT.nc
AQUA_MODIS.20201018T231501.L2.SST.NRT.nc
AQUA_MODIS.20201018T232001.L2.SST.NRT.nc
AQUA_MODIS.20201018T232501.L2.SST.NRT.nc
AQUA_MODIS.20201019T005001.L2.SST.NRT.nc
AQUA_MODIS.20201019T005501.L2.SST.NRT.nc
AQUA_MODIS.20201019T010001.L2.SST.NRT.nc
AQUA_MODIS.20201019T204500.L2.SST.NRT.nc
AQUA_MODIS.20201019T205000.L2.SST.NRT.nc
AQUA_MODIS.20201019T221501.L2.SST.NRT.nc
AQUA_MODIS.20201019T222001.L2.SST.NRT.nc
AQUA_MODIS.20201019T222500.L2.SST.NRT.nc
AQUA_MODIS.20201019T223000.L2.SST.NRT.nc
AQUA_MODIS.20201019T235501.L2.SST.NRT.nc
AQUA_MODIS.20201020T000001.L2.SST.NRT.nc
AQUA_MODIS.20201020T000500.L2.SST.NRT.nc
AQUA_MODIS.20201020T013501.L2.SST.NRT.nc
AQUA_MODIS.20201020T014001.L2.SST.NRT.nc
AQUA_MODIS.20201020T014500.L2.SST.NRT.nc
AQUA_MODIS.20201020T195501.L2.SST.NRT.nc
AQUA_MODIS.20201020T212000.L2.SST.NRT.nc
AQUA_MODIS.20201020T212501.L2.SST.NRT.nc
AQUA_MODIS.20201020T213001.L2.SST.NRT.nc
AQUA_MODIS.20201020T230000.L2.SST.NRT.nc
AQUA_MODIS.20201020T230501.L2.SST.NRT.nc
AQUA_MODIS.20201020T231001.L2.SST.NRT.nc
AQUA_MODIS.20201021T004000.L2.SST.NRT.nc
AQUA_MODIS.20201021T004500.L2.SST.NRT.nc
AQUA_MODIS.20201021T005001.L2.SST.NRT.nc
AQUA_MODIS.20201021T203501.L2.SST.NRT.nc
AQUA_MODIS.20201021T220501.L2.SST.NRT.nc
AQUA_MODIS.20201021T221001.L2.SST.NRT.nc
AQUA_MODIS.20201021T221501.L2.SST.NRT.nc
AQUA_MODIS.20201021T234001.L2.SST.NRT.nc
AQUA_MODIS.20201021T234501.L2.SST.NRT.nc
AQUA_MODIS.20201021T235001.L2.SST.NRT.nc
AQUA_MODIS.20201021T235501.L2.SST.NRT.nc
AQUA_MODIS.20201022T012001.L2.SST.NRT.nc
AQUA_MODIS.20201022T012501.L2.SST.NRT.nc
AQUA_MODIS.20201022T013001.L2.SST.NRT.nc
AQUA_MODIS.20201022T013501.L2.SST.NRT.nc

oceani
Posts: 33
Joined: Wed Aug 23, 2006 12:32 pm America/New_York

Subscriptions Empty

by oceani » Thu Oct 22, 2020 12:49 pm America/New_York

Hi John,

We have been using a user name and password for many years, but I gave your wget version a shot anyways.  Using the command below, I get the following results:

$ wget --post-data="subID=906&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search

--2020-10-22 10:41:38--  https://oceandata.sci.gsfc.nasa.gov/api/file_search
Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx
Connecting to oceandata.sci.gsfc.nasa.gov|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: “STDOUT”

    [ <=>                                                       ] 0           --.-K/s   in 0s     

    2020-10-22 10:41:39 (0.00 B/s) - written to stdout [0/0]

If I do the same thing and direct the result to a file, the file comes up empty too.

Strange that it works for you and not on this end.  Note - All has been working fine for our many subscriptions until sometime early this morning.  This is happening on two independent machines which have not been touched.  We are now ( as of the last 1-2 hours or so) seeing the same problem for our CHL subscriptions as well.

Thanks for your help.

Mark

OB ODPS - jgwilding
Subject Matter Expert
Subject Matter Expert
Posts: 110
Joined: Fri Feb 19, 2021 1:09 pm America/New_York

Subscriptions Empty

by OB ODPS - jgwilding » Thu Oct 22, 2020 1:41 pm America/New_York

Mark,

We think you might have gotten an IP block.  Can you post your public-facing IP address?

john

oceani
Posts: 33
Joined: Wed Aug 23, 2006 12:32 pm America/New_York

Subscriptions Empty

by oceani » Thu Oct 22, 2020 1:46 pm America/New_York

xx.xxx.xx.xx

Any idea why our IP address was blocked?

Mark

OB ODPS - jgwilding
Subject Matter Expert
Subject Matter Expert
Posts: 110
Joined: Fri Feb 19, 2021 1:09 pm America/New_York

Subscriptions Empty

by OB ODPS - jgwilding » Thu Oct 22, 2020 1:51 pm America/New_York

Thanks.  I've forwarded it to our systems group.  They'll check for a block.  Usually a block occurs if the traffic exceeds certain limits, i.e. too many concurrent connections, too many attempts to get non-existent files.

john

oceani
Posts: 33
Joined: Wed Aug 23, 2006 12:32 pm America/New_York

Subscriptions Empty

by oceani » Thu Oct 22, 2020 3:59 pm America/New_York

John,

That must have done the trick.  Looks like we are getting our subscription files again.

Thanks for your help!

Mark

OB ODPS - jgwilding
Subject Matter Expert
Subject Matter Expert
Posts: 110
Joined: Fri Feb 19, 2021 1:09 pm America/New_York

Subscriptions Empty

by OB ODPS - jgwilding » Thu Oct 22, 2020 6:06 pm America/New_York

Glad to hear that.  However, our systems group did not see any blocks for that IP, so i'm not sure what was going on.

john

oceani
Posts: 33
Joined: Wed Aug 23, 2006 12:32 pm America/New_York

Subscriptions Empty

by oceani » Fri Oct 23, 2020 1:13 pm America/New_York

John,

Strange.  This morning some of our subscription list requests returned complete file listings with the wget commands and other subscriptions returned no information (zero files listed - not even older data) using the same commands (just different subscription ID numbers).  Not sure what is happening.  Looks like your system was down earlier today (could not even get to your website/forum).  Perhaps this is related.

Thanks for any further assistance.

Mark

OB ODPS - jgwilding
Subject Matter Expert
Subject Matter Expert
Posts: 110
Joined: Fri Feb 19, 2021 1:09 pm America/New_York

Subscriptions Empty

by OB ODPS - jgwilding » Fri Oct 23, 2020 1:25 pm America/New_York

Mark,

Apparently not being able to access our web site and forum is an indicator of an IP block.  The blocks are dynamically added and removed.  To my knowledge, nothing was down this morning.  Are you running the queries for the different subscriptions concurrently or just one at a time?

john

Post Reply