curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/G20161220016.L1B_COMS.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20150501001642.he5.gz
I am always successful with CentOS 6 machines. I have had a success or two with 3 different CentOS 7 machines.
The failures look like this:
searobin 182 :/tmp/test> curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/G20161220016.L1B_COMS.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20150501001642.he5.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
...
0 0 0 0 0 0 0 0 --:--:-- 0:00:13 --:--:-- 0
0 0 0 193 0 0 13 0 --:--:-- 0:00:14 --:--:-- 13
curl: (47) Maximum (50) redirects followed
The next time I happened to catch it correctly somehow (which i cntl-C out of when the data started to d/l)
searobin 183 :/tmp/test> curl --ipv4 -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/G20161220016.L1B_COMS.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20150501001642.he5.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 193 0 0 143 0 --:--:-- 0:00:01 --:--:-- 143
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
2 734M 2 16.2M 0 0 4431k 0 0:02:49 0:00:03 0:02:46 11.6M^C
One one CentOS machine there is no joy ever...
seahawk 125 :/optics1/home1/oo_processing> curl --verbose -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/G20161220016.L1B_COMS.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20150501001642.he5.gz
* Couldn't find host oceandata.sci.gsfc.nasa.gov in the .netrc file; using defaults
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to oceandata.sci.gsfc.nasa.gov port 443 (#0)
* Trying xx.xxx.xx.xx...
0 0 0 0 0 0 0 0 --:--:-- 0:02:07 --:--:-- 0* Connection timed out
* Trying 2001:4d0:2418:128::84...
* Failed to connect to 2001:4d0:2418:128::84: Network is unreachable
* Failed connect to oceandata.sci.gsfc.nasa.gov:443; Network is unreachable
* Closing connection 0
curl: (7) Failed to connect to 2001:4d0:2418:128::84: Network is unreachable
I notice that the machine seems to be trying ipv6? Why would this one machine be resolving to an ipv6 number?
So I forced it to use ipv4:
seahawk 126 :/optics1/home1/oo_processing> curl -ipv4 --verbose -b ~/.urs_cookies -c ~/.urs_cookies -L -n --output /tmp/G20161220016.L1B_COMS.he5.gz https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/COMS_GOCI_L1B_GA_20150501001642.he5.gz
* Couldn't find host oceandata.sci.gsfc.nasa.gov in the .netrc file; using defaults
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to oceandata.sci.gsfc.nasa.gov port 443 (#0)
* Trying xx.xxx.xx.xx...
0 0 0 0 0 0 0 0 --:--:-- 0:02:07 --:--:-- 0* Connection timed out
* Failed connect to oceandata.sci.gsfc.nasa.gov:443; Connection timed out
* Closing connection 0
curl: (7) Failed connect to oceandata.sci.gsfc.nasa.gov:443; Connection timed out
And yet this machine can reach google just fine:
seahawk 127 :/optics1/home1/oo_processing> ping google.com
PING google.com (xx.xxx.xx.xx) 56(84) bytes of data.
64 bytes from xx.xxx.xx.xx (xx.xxx.xx.xx): icmp_seq=1 ttl=59 time=6.81 ms
64 bytes from xx.xxx.xx.xx (xx.xxx.xx.xx): icmp_seq=2 ttl=59 time=6.80 ms
64 bytes from xx.xxx.xx.xx (xx.xxx.xx.xx): icmp_seq=3 ttl=59 time=6.82 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 6.806/6.812/6.820/0.067 ms
And curl seems to be working just fine!
seahawk 128 :/optics1/home1/oo_processing> curl --verbose --output installing.html http://matplotlib.org/basemap/users/installing.html
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to matplotlib.org port 80 (#0)
* Trying xx.xxx.xx.xx...
* Connected to matplotlib.org (xx.xxx.xx.xx) port 80 (#0)
> GET /basemap/users/installing.html HTTP/1.1
> User-Agent: curl/7.29.0
> Host: matplotlib.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: GitHub.com
< Date: Thu, 30 Mar 2017 00:41:17 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 10012
< Last-Modified: Fri, 14 Feb 2014 19:25:23 GMT
< Access-Control-Allow-Origin: *
< Expires: Thu, 30 Mar 2017 00:51:17 GMT
< Cache-Control: max-age=600
< Accept-Ranges: bytes
< X-GitHub-Request-Id: CB46:5914:44A2DB:5DDD5B:58DC542D
<
{ [data not shown]
100 10012 100 10012 0 0 118k 0 --:--:-- --:--:-- --:--:-- 119k
* Connection #0 to host matplotlib.org left intact
seahawk 129 :/optics1/home1/oo_processing> ll -h installing.html
-rw-rw-r-- 1 oo_processing oo_processing 9.8K Mar 29 20:41 installing.html
Ideas? I have a feeling it has to do with the down time today with your servers? I am attaching the redirect errors from a "curl --trace" and a "curl --trace-ascii" version of the above.
Thanks,
Brockattachment 1
attachment 2