getfile problem

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
alaroy
Posts: 65
Joined: Tue Nov 21, 2006 3:33 pm America/New_York

getfile problem

by alaroy » Wed Oct 23, 2019 10:46 pm America/New_York

Since around 2019-10-23 20:43:16 UTC any attempt to retrieve a file using the getfile cgi has been failing with an "Error 404 not found". Could somebody please investigate the problem? Thank you.

Tags:

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 142
Joined: Tue Feb 09, 2021 8:19 am America/New_York

getfile problem

by OB WebDev - norman » Thu Oct 24, 2019 10:31 am America/New_York

Hi,

Could you please give us some additional information?
You will find some instructions here.

https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=6447

Thanks,
Norman

alaroy
Posts: 65
Joined: Tue Nov 21, 2006 3:33 pm America/New_York

getfile problem

by alaroy » Thu Oct 24, 2019 10:36 am America/New_York

Norman,

Thank you for your response. The problem seems to have been resolved shortly before 10:26 UTC (6:26 EDT).

-Andrew L.

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 142
Joined: Tue Feb 09, 2021 8:19 am America/New_York

getfile problem

by OB WebDev - norman » Thu Oct 24, 2019 10:37 am America/New_York

OK. Great.

Norman

b.detracey
Posts: 38
Joined: Tue Sep 29, 2015 11:10 am America/New_York

getfile problem

by b.detracey » Thu Oct 24, 2019 2:07 pm America/New_York

Having the same problem here:
$ wget -v --content-disposition 'https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/20190101120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc'
--2019-10-24 15:04:25--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/20190101120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-10-24 15:05:16 ERROR 404: Not Found.

oo_processing
Posts: 240
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

getfile problem

by oo_processing » Thu Oct 24, 2019 2:49 pm America/New_York

I had the same problem last night near midnight where all my VIIRS and MODIS downloaded files were "bad" as in html files.
It happened on machines in both Tampa and in Saint Petersburg.

I was able to d/l the correct files doy 296, today.
However, I am getting the errors again just now (note the 665 file size which is the error message) and where get.txt contains:
[bmurch@optics1] /cms_zfs/sat_data/modis/l0/2019/297 $ cat get.txt
MOD00.A2019297.1015_1.PDS.bz2
MOD00.A2019297.1020_1.PDS.bz2
MOD00.A2019297.1025_1.PDS.bz2
MOD00.A2019297.1150_1.PDS.bz2
MOD00.P2019297.1455_1.PDS.bz2
MOD00.P2019297.1500_1.PDS.bz2
MOD00.P2019297.1540_1.PDS.bz2
MOD00.P2019297.1640_1.PDS.bz2

And I killed it after the three html files were downloaded.

[bmurch@optics1] /cms_zfs/sat_data/modis/l0/2019/297 $ time curl --interface 2607:fe50:0:6330::132 --retry 5 --retry-delay 2 --max-time 0 --remote-name-all https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/{$(sed ':a;N;$!ba;s/\n/,/g' get.txt)}

[1/8]: https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1015_1.PDS.bz2 --> MOD00.A2019297.1015_1.PDS.bz2
--_curl_--https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1015_1.PDS.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
110   665 110   665    0     0     99      0  0:00:06  0:00:06 --:--:--   194

[2/8]: https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1020_1.PDS.bz2 --> MOD00.A2019297.1020_1.PDS.bz2
--_curl_--https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1020_1.PDS.bz2
110   665 110   665    0     0    122      0  0:00:05  0:00:05 --:--:--   150

[3/8]: https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1025_1.PDS.bz2 --> MOD00.A2019297.1025_1.PDS.bz2
--_curl_--https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1025_1.PDS.bz2
110   665  110   665    0     0    122      0  0:00:05  0:00:05 --:--:--   150

[4/8]: https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1150_1.PDS.bz2 --> MOD00.A2019297.1150_1.PDS.bz2
--_curl_--https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.A2019297.1150_1.PDS.bz2
  0   665    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0^C

real    0m19.311s
user    0m0.044s
sys     0m0.056s

oo_processing
Posts: 240
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

getfile problem

by oo_processing » Thu Oct 24, 2019 3:15 pm America/New_York

It is working again.
If I had to guess, I suspect that the real-time subscription database and the other one are out of sync.

OB WebDev - dana.r.wilson
Subject Matter Expert
Subject Matter Expert
Posts: 54
Joined: Mon Apr 20, 2020 8:04 am America/New_York

getfile problem

by OB WebDev - dana.r.wilson » Thu Oct 24, 2019 3:24 pm America/New_York

The issues have been fixed.

oo_processing
Posts: 240
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

getfile problem

by oo_processing » Thu Oct 24, 2019 4:14 pm America/New_York

I have had another 404 error "file not found" just now. I wonder if the file is on your servers, as other files can be downloaded now:
wget https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A2003275023500.L0_LAC.bz2
--2019-10-24 16:09:25--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A2003275023500.L0_LAC.bz2
Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-10-24 16:09:31 ERROR 404: Not Found.

oo_processing
Posts: 240
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

getfile problem

by oo_processing » Thu Oct 24, 2019 4:28 pm America/New_York

I have tried the other possible names without success (although Chuanmin Hu tired A2003275023500.L0_LAC.bz2 earlier and it worked fine):

From this page:
https://oceancolor.sci.gsfc.nasa.gov/cgi/browse.pl/filelist.1571948228.txt?sub=filenamelist&id=1571948228.23574&prm=L0
I found MOD00.P2003275.0235_1.PDS
and then tried that as well:

what 117 :/optics1/home1/oo_processing> wget https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.P2003275.0235_1.PDS.bz2
--2019-10-24 16:21:53--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.P2003275.0235_1.PDS.bz2
Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-10-24 16:21:59 ERROR 404: Not Found.

what 118 :/optics1/home1/oo_processing> wget https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.P2003275.0235_1.PDS
--2019-10-24 16:22:07--  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.P2003275.0235_1.PDS
Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov|xx.xxx.xx.xx|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-10-24 16:22:13 ERROR 404: Not Found

Post Reply