Files not able to download (And missing GOCI L1 Files)

Please enter here to ask a question about any NASA Science related topics!
Post Reply
oo_processing
Posts: 208
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

Files not able to download (And missing GOCI L1 Files)

by oo_processing » Sun Apr 02, 2017 5:48 pm America/New_York

https://oceandata.sci.gsfc.nasa.gov/search/file_search.cgi I am able to find this file and download in a browser.
https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20160507051641.he5.gz
curl --verbose -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/COMS_GOCI_L1B_GA_20160507051641.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20160507051641.he5.gz
* Couldn't find host oceandata.sci.gsfc.nasa.gov in the .netrc file; using defaults
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to oceandata.sci.gsfc.nasa.gov port 443 (#0)
*   Trying xx.xxx.xx.xx...
* Connected to oceandata.sci.gsfc.nasa.gov (xx.xxx.xx.xx) port 443 (#0)
...

> Accept: */*
> Cookie: app-obdaac=13232cb964b44c7f551e8dd9d5d85f6066d53051
>


  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0< HTTP/1.1 404 Not Found

The same is true of this file that the file search page returns:
Filename  Last Modified  Size  
COMS_GOCI_L1B_GA_20160514011641.he5.gz  2016-05-16 05:57:00  750927.56 (kB)

curl --verbose -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/G20161220016.L1B_COMS.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20160514011641.he5.gz

searobin 207 :/tmp/test> curl --verbose -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/COMS_GOCI_L1B_GA_20160514011641.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20160514011641.he5.gz
* Couldn't find host oceandata.sci.gsfc.nasa.gov in the .netrc file; using defaults
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to oceandata.sci.gsfc.nasa.gov port 443 (#0)
*   Trying xx.xxx.xx.xx...
* Connected to oceandata.sci.gsfc.nasa.gov (xx.xxx.xx.xx) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
...

>


< HTTP/1.1 302 Found
< Server: nginx
< Date: Sun, 02 Apr 2017 21:31:08 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< Location: /goci/getfile/COMS_GOCI_L1B_GA_20160514011641.he5.gz
...

> Accept: */*
> Cookie: app-obdaac=3bbe3fa9a65ad3c5cc762e75fac74addd3023b18
>


  0     0    0     0    0     0      0      0 --:--:--  0:00:11 --:--:--     0< HTTP/1.1 404 Not Found

However, I receive the error html in the outfile for both:

There are L1 files missing as well. I searched for May, 1 2016 and find the L2, but no L1 file as there are for most files:
eg this search:
https://oceancolor.gsfc.nasa.gov/cgi/browse.pl?sub=level1or2&per=DAY&day=16922&prm=CHL&set=10&mon=16922&sen=g1&rad=0&frc=0&n=28.4765625&w=131.1328125&id=1491169338.17140&cn=29.19&cs=28.03&cw=129.63&ce=131.47&dnm=D&file=G2016122021641.L2_COMS_OC_CHLOR_A_BRS&ndx=5&wid=796&hgt=813

returns a link to this file only
https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/G2016122021641.L2_COMS_OC.nc

Whereas this search:
https://oceancolor.gsfc.nasa.gov/cgi/browse.pl?sub=level1or2&per=DAY&day=16913&prm=CHL&set=10&mon=16892&sen=g1&rad=0&frc=0&n=34.1015625&w=129.7265625&id=1491169579.17816&cn=35.83&cs=33.87&cw=128.93&ce=130.81&dnm=D&file=G2016113071640.L2_COMS_OC_CHLOR_A_BRS&ndx=0&wid=796&hgt=813

gives both
https://oceandata.sci.gsfc.nasa.gov/goci/getfile/COMS_GOCI_L1B_GA_20160422071640.he5.gz
and
https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/G2016113071640.L2_COMS_OC.nc

Why are there L1 files missing for the GOCI data?

Thanks,
Brock

Tags:

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 158
Joined: Fri Feb 05, 2021 9:17 am America/New_York

Files not able to download (And missing GOCI L1 Files)

by OB ODPS - towens » Sun Apr 02, 2017 6:37 pm America/New_York

We lost a back-end file storage device on Friday evening.  Try your file request on Monday after 10am EDT.  The host should be back online by then.

Tommy

huhuhu
Posts: 14
Joined: Fri Jun 25, 2010 11:46 am America/New_York

Files not able to download (And missing GOCI L1 Files)

by huhuhu » Mon Apr 03, 2017 6:31 pm America/New_York

Thanks Tommy. But as of 6 pm on 4/3/2017, there are still some files missing? For example on this following link:
https://oceancolor.gsfc.nasa.gov/cgi/browse.pl?sub=level1or2&per=MO&day=16922&prm=CHL&set=10&mon=16922&sen=g1&rad=0&frc=0&n=34.8046875&w=131.8359375&id=1491248254.27586&cn=35.34&cs=33.37&cw=130.07&ce=131.95&dnm=D&file=G2016122001641.L2_COMS_OC_CHLOR_A_BRS&ndx=247&wid=796&hgt=813

Only L2 file is there, but L1 file is not. There are quite some cases like this. Is this random or there are just some files in a certain period missing?
Regards, Chuanmin

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 158
Joined: Fri Feb 05, 2021 9:17 am America/New_York

Files not able to download (And missing GOCI L1 Files)

by OB ODPS - towens » Mon Apr 03, 2017 8:52 pm America/New_York

I am seeing the L1 files now. 
Try refreshing your browser.

Tommy

huhuhu
Posts: 14
Joined: Fri Jun 25, 2010 11:46 am America/New_York

Files not able to download (And missing GOCI L1 Files)

by huhuhu » Tue Apr 04, 2017 9:06 am America/New_York

Yes that file now shows up and can be downloaded either manually or using curl command line.

But last night I could download the following files using curl:
G20161400416    Unable to download file
G20161400516    Unable to download file
G20161400616    Unable to download file
G20161400716    Unable to download file
G20161410016    Unable to download file
G20161410116    Unable to download file

This morning I tried again and I could download them using curl. It appears that this problem is intermittent. I'll see if it continues. Thanks! Chuanmin

huhuhu
Posts: 14
Joined: Fri Jun 25, 2010 11:46 am America/New_York

Files not able to download (And missing GOCI L1 Files)

by huhuhu » Tue Apr 04, 2017 3:22 pm America/New_York

So far so good. Chuanmin

Post Reply