Earthdata Forum

All of the queries for my subscriptions are coming up empty as of this morning.  These are mostly subscriptions for Level 2 MODIS (both Terra and Aqua) and VIIRS SST data.  We have several subscriptions, but a few subscription numbers for reference are 906, 2650 and 2642.  The initial command that we use to see the content of our subscription file listing is:
wget --user=XXXXX --password=XXXXXXX --post-data="subID=906&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search
This command is coming up with zero results for most-to-all of our SST subscriptions. 

Also when I click on the File Search button (far right column) on View Active-->My Subscriptions page it seems to be oscillating between there being no entries to being a full list of data files.  For the past 15-20 minutes or so these listings have been completely empty for all of my subscriptions.

Please advise.

Thanks much,

Mark

Mark,

I don't think you need the user/passwd options for the file search.  When i run this, i get files through today (Oct 22).  The 2> redirects the wget msgs to /dev/null.  Does this work for you?

john

$ wget --post-data="subID=906&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search 2> /dev/null
AQUA_MODIS.20201018T200501.L2.SST.NRT.nc
AQUA_MODIS.20201018T213501.L2.SST.NRT.nc
AQUA_MODIS.20201018T214001.L2.SST.NRT.nc
AQUA_MODIS.20201018T214501.L2.SST.NRT.nc
AQUA_MODIS.20201018T231001.L2.SST.NRT.nc
AQUA_MODIS.20201018T231501.L2.SST.NRT.nc
AQUA_MODIS.20201018T232001.L2.SST.NRT.nc
AQUA_MODIS.20201018T232501.L2.SST.NRT.nc
AQUA_MODIS.20201019T005001.L2.SST.NRT.nc
AQUA_MODIS.20201019T005501.L2.SST.NRT.nc
AQUA_MODIS.20201019T010001.L2.SST.NRT.nc
AQUA_MODIS.20201019T204500.L2.SST.NRT.nc
AQUA_MODIS.20201019T205000.L2.SST.NRT.nc
AQUA_MODIS.20201019T221501.L2.SST.NRT.nc
AQUA_MODIS.20201019T222001.L2.SST.NRT.nc
AQUA_MODIS.20201019T222500.L2.SST.NRT.nc
AQUA_MODIS.20201019T223000.L2.SST.NRT.nc
AQUA_MODIS.20201019T235501.L2.SST.NRT.nc
AQUA_MODIS.20201020T000001.L2.SST.NRT.nc
AQUA_MODIS.20201020T000500.L2.SST.NRT.nc
AQUA_MODIS.20201020T013501.L2.SST.NRT.nc
AQUA_MODIS.20201020T014001.L2.SST.NRT.nc
AQUA_MODIS.20201020T014500.L2.SST.NRT.nc
AQUA_MODIS.20201020T195501.L2.SST.NRT.nc
AQUA_MODIS.20201020T212000.L2.SST.NRT.nc
AQUA_MODIS.20201020T212501.L2.SST.NRT.nc
AQUA_MODIS.20201020T213001.L2.SST.NRT.nc
AQUA_MODIS.20201020T230000.L2.SST.NRT.nc
AQUA_MODIS.20201020T230501.L2.SST.NRT.nc
AQUA_MODIS.20201020T231001.L2.SST.NRT.nc
AQUA_MODIS.20201021T004000.L2.SST.NRT.nc
AQUA_MODIS.20201021T004500.L2.SST.NRT.nc
AQUA_MODIS.20201021T005001.L2.SST.NRT.nc
AQUA_MODIS.20201021T203501.L2.SST.NRT.nc
AQUA_MODIS.20201021T220501.L2.SST.NRT.nc
AQUA_MODIS.20201021T221001.L2.SST.NRT.nc
AQUA_MODIS.20201021T221501.L2.SST.NRT.nc
AQUA_MODIS.20201021T234001.L2.SST.NRT.nc
AQUA_MODIS.20201021T234501.L2.SST.NRT.nc
AQUA_MODIS.20201021T235001.L2.SST.NRT.nc
AQUA_MODIS.20201021T235501.L2.SST.NRT.nc
AQUA_MODIS.20201022T012001.L2.SST.NRT.nc
AQUA_MODIS.20201022T012501.L2.SST.NRT.nc
AQUA_MODIS.20201022T013001.L2.SST.NRT.nc
AQUA_MODIS.20201022T013501.L2.SST.NRT.nc

Hi John,

We have been using a user name and password for many years, but I gave your wget version a shot anyways.  Using the command below, I get the following results:

$ wget --post-data="subID=906&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search

--2020-10-22 10:41:38--  https://oceandata.sci.gsfc.nasa.gov/api/file_search
Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx
Connecting to oceandata.sci.gsfc.nasa.gov|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/plain]
Saving to: “STDOUT”

    [ <=>                                                       ] 0           --.-K/s   in 0s     

    2020-10-22 10:41:39 (0.00 B/s) - written to stdout [0/0]

If I do the same thing and direct the result to a file, the file comes up empty too.

Strange that it works for you and not on this end.  Note - All has been working fine for our many subscriptions until sometime early this morning.  This is happening on two independent machines which have not been touched.  We are now ( as of the last 1-2 hours or so) seeing the same problem for our CHL subscriptions as well.

Thanks for your help.

Mark

Mark,

We think you might have gotten an IP block.  Can you post your public-facing IP address?

john

xx.xxx.xx.xx

Any idea why our IP address was blocked?

Mark

Thanks.  I've forwarded it to our systems group.  They'll check for a block.  Usually a block occurs if the traffic exceeds certain limits, i.e. too many concurrent connections, too many attempts to get non-existent files.

john

John,

That must have done the trick.  Looks like we are getting our subscription files again.

Thanks for your help!

Mark

Glad to hear that.  However, our systems group did not see any blocks for that IP, so i'm not sure what was going on.

john

John,

Strange.  This morning some of our subscription list requests returned complete file listings with the wget commands and other subscriptions returned no information (zero files listed - not even older data) using the same commands (just different subscription ID numbers).  Not sure what is happening.  Looks like your system was down earlier today (could not even get to your website/forum).  Perhaps this is related.

Thanks for any further assistance.

Mark

Mark,

Apparently not being able to access our web site and forum is an indicator of an IP block.  The blocks are dynamically added and removed.  To my knowledge, nothing was down this morning.  Are you running the queries for the different subscriptions concurrently or just one at a time?

john