Page 2 of 4

wget redirection

Posted: Thu May 21, 2020 9:47 pm America/New_York
by OB ODPS - jgwilding
There was supposed to be network maintenance going on between 17:00-00:00 EDT that could cause intermittent connectivity problems.

john

wget redirection

Posted: Thu May 21, 2020 10:53 pm America/New_York
by emaure
Just an additional information

Trying direct download through the browser (chrome), redirection return error:
              .:. ERROR .:.
OceanColor Biology Processing Group (OBPG)
Sorry, an error has occurred. Use the back button to return to the previous page or go to the Ocean Color Home Page.


Maybe the maintenance explains the errors but if not, I am adding the above hoping that it will be useful.

wget redirection

Posted: Fri May 22, 2020 9:10 am America/New_York
by oo_processing
Note that the problem exists for anc data as well
[bmurch@optics0 ~]$ ll /shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0???.003
-rw-rw-r-- 1 bmurch cms_optics 665 May 22 00:13 /shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0200.003
-rw-rw-r-- 1 bmurch cms_optics 665 May 22 00:13 /shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0400.003
-rw-rw-r-- 1 bmurch cms_optics 665 May 22 00:13 /shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0600.003
-rw-rw-r-- 1 bmurch cms_optics 665 May 22 00:13 /shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0800.003

/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0200.003:     HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0400.003:      HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0600.003:      HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.0800.003:     HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.1000.003:      HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.1200.003:      HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.1400.003:      HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.1600.003:      data                                               
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.1800.003:     HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.2000.003:      HTML document text                                 
/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/var/anc/2020/138/PM1ATTNR.P2020138.2200.003:      HTML document text

wget redirection

Posted: Fri May 22, 2020 9:36 am America/New_York
by OB.DAAC - SeanBailey
I don't have confirmation that the issue was related to the maintenance done on the authentication servers...but the problem did seem to start about that time...
A minor change to how we connect to them has been put in place and seems to have fixed the issue...wondering if it fixes the previous max redirects error as well.

Try your downloads again and let me know if it's still an issue for you.

Sean

wget redirection

Posted: Fri May 22, 2020 9:46 am America/New_York
by bruce
Well I just tried my automated download and I still get redirection problems...

B

wget redirection

Posted: Fri May 22, 2020 9:47 am America/New_York
by OB.DAAC - SeanBailey
...but not a 403 error?

wget redirection

Posted: Fri May 22, 2020 10:16 am America/New_York
by bruce
Just the redirection error (but I never got a 403 error, so from my perspective, no change)

wget redirection

Posted: Fri May 22, 2020 10:19 am America/New_York
by OB.DAAC - SeanBailey
Well,  at least the situation didn't get worse :wink:
Guess we still have some forensics to do...

wget redirection

Posted: Fri May 22, 2020 11:34 am America/New_York
by alaroy
For whatever it's worth our downloads seem to be back to normal starting about 13:00 UTC (aka 9 EDT) today

wget redirection

Posted: Fri May 22, 2020 11:46 am America/New_York
by oo_processing
I have the same problem with redirections.

< HTTP/1.1 302 Found                                                                                                                                                              
< Server: nginx                                                                                                                                                                   
< Date: Fri, 22 May 2020 15:25:33 GMT                                                                                                                                             
< Transfer-Encoding: chunked                                                                                                                                                      
< Connection: keep-alive                                                                                                                                                          
< Keep-Alive: timeout=60                                                                                                                                                          
< Location: https://urs.earthdata.nasa.gov/oauth/authorize?client_id=Z0u-MdLNypXBjiDREZ3roA&response_type=code&redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict                                                                                                                                                                      
< Expires: Mon, 01 Jan 1970 00:00:00 GMT                                                                                                                                          
< Cache-Control: no-cache, must-revalidate, max-age=0, no-store                                                                                                                   
< Pragma: no-cache                                                                                                                                                                
* Replaced cookie app-obdaac="880e0858d5c227ef26f66a3c8c0344c38f0c9875" for domain oceandata.sci.gsfc.nasa.gov, path /, expire 0                                                  
< Set-Cookie: app-obdaac=880e0858d5c227ef26f66a3c8c0344c38f0c9875; path=/; secure                                                                                                 
< Referrer-Policy: no-referrer                                                                                                                                                    
< Expect-CT: max-age=31536000, enforce                                                                                                                                            
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload                                                                                                         
< Content-Security-Policy: upgrade-insecure-requests; default-src 'self' oceancolor.gsfc.nasa.gov data:; script-src 'self' 'unsafe-inline' 'unsafe-eval' www.google-analytics.com www.googletagmanager.com cdn.earthdata.nasa.gov dap.digitalgov.gov data:; style-src 'self' 'unsafe-inline' code.jquery.com cdn.earthdata.nasa.gov; img-src 'self' data: oceancolor.gsfc.nasa.gov www.google-analytics.com cdn.earthdata.nasa.gov                                                                                                                      
<                                                                                                                                                                                 
* Ignoring the response-body                                                                                                                                                      
{ [data not shown]                                                                                                                                                                
  0     0    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--     0* Connection #0 to host oceandata.sci.gsfc.nasa.gov left intact                                     
* Issue another request to this URL: 'https://urs.earthdata.nasa.gov/oauth/authorize?client_id=Z0u-MdLNypXBjiDREZ3roA&response_type=code&redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict'                                                                                                                                           
* Re-using existing connection! (#1) with host urs.earthdata.nasa.gov
* Connected to urs.earthdata.nasa.gov (2001:4d0:241a:4081::89) port 443 (#1)
* Server auth using Basic with user 'oo_processing'

> GET /oauth/authorize?client_id=Z0u-MdLNypXBjiDREZ3roA&response_type=code&redirect_uri=https%3A%2F%2Foceandata.sci.gsfc.nasa.gov%2Fob%2Fgetfile%2Frestrict HTTP/1.1
> Authorization: Basic b29fcHJvY2Vzc2luZzpJZzJpd2tzRkpVaHY=
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.11 libidn/1.18 libssh2/1.4.2
> Host: urs.earthdata.nasa.gov
> Accept: */*
> Cookie: _urs-gui_session=470e1acb2262074a485b8519e88c629a; urs_user_already_logged=yes
>


< HTTP/1.1 302 Found
< Server: nginx/1.17.5
< Date: Fri, 22 May 2020 15:25:33 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: strict-origin-when-cross-origin
< Access-Control-Allow-Origin: null
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Methods: GET, POST
< Access-Control-Expose-Headers: true
< Location: https://oceandata.sci.gsfc.nasa.gov/ob/getfile/restrict?code=37602330dd13da46df4ff6d4fd0dff6aff91e3694c68e428c649a2b9d6ee5173
< Cache-Control: no-cache
* Replaced cookie urs_user_already_logged="yes" for domain earthdata.nasa.gov, path /, expire 1590247533
< Set-Cookie: urs_user_already_logged=yes; domain=earthdata.nasa.gov; path=/; expires=Sat, 23 May 2020 15:25:33 GMT
* Replaced cookie _urs-gui_session="470e1acb2262074a485b8519e88c629a" for domain urs.earthdata.nasa.gov, path /, expire 1590247533
< Set-Cookie: _urs-gui_session=470e1acb2262074a485b8519e88c629a; path=/; expires=Sat, 23 May 2020 15:25:33 GMT; HttpOnly
< X-Request-Id: 8efb344a-907d-4803-9410-70715f4df3ff
< X-Runtime: 0.098615
< Strict-Transport-Security: max-age=31536000
<
* Ignoring the response-body
{ [data not shown]
191   191    0   191    0     0     28      0 --:--:--  0:00:06 --:--:--    28* Connection #1 to host urs.earthdata.nasa.gov left intact
* Maximum (50) redirects followed

curl: (47) Maximum (50) redirects followed
* Closing connection #0
* Closing connection #1
$curl_command_returns: