New thread on manual update of OCSSW processors

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
asubramaniam
Posts: 10
Joined: Thu Aug 11, 2005 3:53 pm America/New_York
Answers: 0

New thread on manual update of OCSSW processors

by asubramaniam » Tue Jan 17, 2017 2:41 pm America/New_York

I am starting a new thread to bring the discussion back to the manual update:

1) Does the python script work with python 3.5.2 or does it need 2.7.8?
I downloaded the install_ocssw.py using the link that Sean had posted but it dies on me in two different ways:
1) Ajits-MacBook-Pro%./install_ocssw.py --install-dir=$HOME/ocssw --git-branch=v7.3 --aqua --seawifs
  File "./install_ocssw.py", line 51
    print 'Loading checksum file.'
                                 ^
SyntaxError: Missing parentheses in call to 'print'

2) If I force it to use phyton 2.7.8, I get:
/usr/bin/python install_ocssw.py --install-dir=$HOME/ocssw --git-branch=v7.3 --aqua --seawifs
Installing bundles.sha256sum (1 of 15)
--14:39:49--  https://oceandata.sci.gsfc.nasa.gov/ocssw/bundles.sha256sum
           => `bundles.sha256sum'
Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx
Connecting to oceandata.sci.gsfc.nasa.gov[xx.xxx.xx.xx]:443... connected.

Unable to establish SSL connection.

Unable to establish SSL connection.
Error - Executing command "cd /Users/ajit/ocssw; wget --tries=5 --wait=5 https://oceandata.sci.gsfc.nasa.gov/ocssw/bundles.sha256sum"
Bundle checksum file (bundles.sha256sum) not downloaded

I suspect I am doing something really silly
Thanks
cheers
ajit

Tags:

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

New thread on manual update of OCSSW processors

by gnwiii » Wed Jan 18, 2017 6:50 am America/New_York

You aren't doing anything silly at all -- if anyone is being silly it is Apple letting people thing they care about security and then shipping python with an obsolete openssl library.

1) Does the python script work with python 3.5.2 or does it need 2.7.8?

The OCSSW scripts need a python 2.7, but as you see, Apple python (I'm using MacOS El Capitan) uses an obsolete SSL library.  You can get around that by installing python27 and py27-openssl from Macports.  Fink or Homebrew probably work just as well, but NASA has been using Macports gfortran for many years, so we have
Macports on all our MacOS systems.

$ port installed python27 py27-openssl
The following ports are currently installed:
  py27-openssl @16.0.0_0 (active)
  python27 @2.7.13_0 (active)


When you install Macports it adds /opt/local/bin to the front of you path in ~/.profile.
You then need to run "port select python python27".  You can check that everything is in order
as follows:

$ which python
/opt/local/bin/python
$ python -c 'import ssl ; print ssl.OPENSSL_VERSION'
OpenSSL 1.0.2j  26 Sep 2016

jgallen
Posts: 2
Joined: Tue Jul 19, 2016 8:33 am America/New_York
Answers: 0

New thread on manual update of OCSSW processors

by jgallen » Wed Jan 18, 2017 1:11 pm America/New_York

I seem to be having the same issue as well. I got to the same point as you with

$ which python
/usr/local/bin/python
$ python -c 'import ssl ; print ssl.OPENSSL_VERSION'
OpenSSL 1.0.2j  26 Sep 2016


But when I run the ssl check from Sean in another thread:

$ openssl ciphers -tls -v 'HIGH:!ADH:!MD5:@STRENGTH' | egrep "ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384|ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-AES256-SHA384|ECDHE-ECDSA-AES128-SHA256"

I get no response. And I'm right back to where I started with the "unable to establish SSL connection" warning.
So good news, reading all the forum posts and tinkering have been a nice crash course in ssl for me (especially since I'm still very new to all this), but I feel like I'm still running in circles.

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

New thread on manual update of OCSSW processors

by gnwiii » Thu Jan 19, 2017 6:51 am America/New_York

@jgallen

Which MacOS version? How did you install the version of python in /usr/local/bin?  If you used homebrew on El Capitan then your python may be showing the OpenSSL version from the homebrew OpenSSL headers but actually linked to the (insecure, deprecated) Apple library.  See Installing and Running on OS X 10.11 SSL Fails to Link Using Brew #3964 and Update OpenSSL on OS X with Homebrew might apply.  One entry in the stackoverflow thread has openssl installed to a /usr/local/opt/openssl/bin/openssl, so it is possible that the openssl command you are using is not from OpenSSL 1.0.2j.   To check, run which openssl and openssl version.

jgallen
Posts: 2
Joined: Tue Jul 19, 2016 8:33 am America/New_York
Answers: 0

New thread on manual update of OCSSW processors

by jgallen » Thu Jan 19, 2017 2:09 pm America/New_York

> Which MacOS version?


macOS Sierra Version 10.12.2

>How did you install the version of python in /usr/local/bin?


Homebrew. Running which openssl and openssl version confirms that it's still linked to the Apple library (OpenSSL 0.9.8zh 14 Jan 2016 located in /usr/bin/openssl).

Reading through those threads, looks like forcing a link between homebrew OpenSSL and the system library is a no-no due to the potential to break unknown things globally. So I tried the rpath method mentioned here. and noticed my .NET (also installed via Homebrew) was 1.1.0, so a slight change to the script led to:

sudo install_name_tool -add_rpath /usr/local/opt/openssl/lib /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.1.0/System.Security.Cryptography.Native.OpenSsl.dylib

which leads to
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: for: /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.1.0/System.Security.Cryptography.Native.OpenSsl.dylib (for architecture x86_64) option "-add_rpath /usr/local/opt/openssl/lib" would duplicate path, file already has LC_RPATH for: /usr/local/opt/openssl/lib

Forgive me for being new to this, but does that mean it should already be finding the correct library? So the install should work?

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

New thread on manual update of OCSSW processors

by OB.DAAC - SeanBailey » Thu Jan 19, 2017 7:46 pm America/New_York

Another  option is to install a prebuilt python package (e.g. anaconda) .
Works on my Mac...
Sean

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

New thread on manual update of OCSSW processors

by gnwiii » Fri Jan 20, 2017 7:18 am America/New_York

It seems homebrew is not building openssl correctly.  Trying to fix such issues after a package is installed is something for experts, and is more of a quick fix.  Unless homebrew fixes the problem, the workarounds will have to redone for each update.  Anaconda Python (binary packages) and macports (requires Xcode) are known to work with the OCSSW scripts, and both are actively maintained so have a good chance of continuing to work in the future without resorting to workarounds.

Post Reply