Problem installing luts

aleitner · by **aleitner** » Fri Apr 07, 2017 2:59 pm America/New_York

Hello everyone,
I am getting the same error installing luts. I have ensured that I am running the correct python version, I have deleted the OCSSW directory entirely and tried the entire install again, and I have used an ethernet connection. Any help on this error would be greatly appreciated:
Python version acceptable
Installing aqua-luts (12 of 13)
Error! could not establish a network connection. Check your network connection.
If you do not find a problem, please try again later.
Error - Could not install luts for aqua

gnwiii · by **gnwiii** » Fri Apr 07, 2017 9:46 pm America/New_York

If it is any comfort, you are not the only one having this problem. It may be helpful to others if you provide more complete details of your misbehaving configuration. So far I have seen systems (both macOS and linux) with a suitable python installed but the PATH variable set to use a different python, systems with a 3rd party python linked to an obsolete openssl library, and systems with outdated security certificates.

The exact Python version is not critical. What is critical is that the python used by the OCSSW scripts use an SSL library that supports the relatively recent cyphers used by NASA's https servers. Unless you already have macports python, you should probably install Anaconda python, which provides a current SSL library and certificates. 3rd party Python installations sometimes end up using an outdated Apple openssl library or neglect to provide instructions for installing current certificates. You should mention how you installed Python and show us the results of the commands mentioned in Sean's post and also "python -c 'import ssl ; print ssl.OPENSSL_VERSION'".

aleitner · by **aleitner** » Tue Apr 11, 2017 12:35 am America/New_York

Thank you so much for the response.
I am running macOS X 10.11.6.
I have found the exact python version is critical. I have found that python3 does not pass the python version check, nor does the Mac usr version. The only way I can pass the version step is if I am running python2 - specifically python 2.7.13
$ which python
/Users/astridleitner/anaconda3/bin/python #does not work for installs

Because I have and often use python3, I created a separate environment for python2 which I first activate and then use to open the seadas gui with "open -a seadas".

$ which python
/Users/astridleitner/anaconda3/envs/python2/bin/python #passes the version check

$ python --version
Python 2.7.13 :: Continuum Analytics, Inc.

$python -c 'import ssl ; print ssl.OPENSSL_VERSION'
OpenSSL 1.0.2k 26 Jan 2017

Are these all the details and results you need?

Thank you again for your time and help!

gnwiii · by **gnwiii** » Tue Apr 11, 2017 7:37 am America/New_York

You seem to have all the right ingredients. Have you tried running update_luts.py --verbose ... from the command line with the python2 environment? You can also try: /Users/astridleitner/anaconda3/envs/python2/bin/python $OCSSWROOT/run/scripts/update_luts.py --verbose ....

If that works, you can force the scripts to use /Users/astridleitner/anaconda3/envs/python2/bin/python by running pyverchk.py in the python3 environment. pyverchk.py should ask if you have another python installation and then allow you to provide the full path to the desired python. It will then edit the python scripts to force use of your Anaconda python2. This will also allow you to run the SeaDAS GUI normally.

If you can't get update_luts.py to work from the command line, you should check the SSL cyphers from python. Compare this against Sean's list.

aleitner · by **aleitner** » Tue Apr 11, 2017 5:30 pm America/New_York

Thank you for the advice. I think I am missing something basic here, when I run update_luts.py --verbose ... from the command line with the python2 environment, I get this error:
Astrids-MBP:Desktop astridleitner$ source activate python2
(python2) Astrids-MBP:Desktop astridleitner$ update_luts.py --verbose ...
-bash: update_luts.py: command not found
(python2) Astrids-MBP:Desktop astridleitner$ /Users/astridleitner/anaconda3/envs/python2/bin/python $OCSSWROOT/run/scripts/update_luts.py --verbose ....
/Users/astridleitner/anaconda3/envs/python2/bin/python: can't open file '/run/scripts/update_luts.py': [Errno 2] No such file or directory

In my ocssw directory in ocssw/run/scripts I do have the update_luts.py file.
If I set my directory to this folder directly and then try to run update_luts.py:

(python2) Astrids-MBP:scripts astridleitner$ python update_luts.py aqua --verbose
ERROR: OCDATAROOT environment variable not set.

Is my syntax wrong? How do I set the OCDATAROOT environment variable?

For the SSL cyphers, I ran :
(python2) Astrids-MBP:scripts astridleitner$ openssl ciphers -tls -v 'HIGH:!ADH:!MD5:@STRENGTH' | egrep "ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384|ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-AES256-SHA384|ECDHE-ECDSA-AES128-SHA256"
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH    Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH    Au=ECDSA Enc=AES(256) Mac=SHA384
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH    Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH    Au=ECDSA Enc=AES(128) Mac=SHA256

I believe this means I do have the correct cyphers?

On another note, it seems my method of opening the seadas gui from my python2 environment has stopped working! I followed exactly the same protocol, but now my update processors run gets me the version error again! I don't understand how this is possible, since the same code worked a few days ago.
Installing scripts (11 of 13)
File "/Applications/seadas-7.4/ocssw/run/scripts/pyverchk.py", line 26
print "%s does not exist." % ver
   ^
SyntaxError: Missing parentheses in call to 'print'
Error - Python version is not new enough to install luts

If I go back to my terminal and check the python version, however, it shows the proper version!
(python2) Astrids-MBP:scripts astridleitner$ which python
/Users/astridleitner/anaconda3/envs/python2/bin/python
(python2) Astrids-MBP:scripts astridleitner$ python --version
Python 2.7.13 :: Continuum Analytics, Inc.

I am very confused as to what is happening.
Thank you again for all the help!

gnwiii · by **gnwiii** » Tue Apr 11, 2017 6:39 pm America/New_York

For the missing settings of OCSSWROOT and OCDATAROOT, see the very bottom section of SeaDAS Installion:

"Add the following lines to your “.bashrc” file to configure your command line enviroment [sic]. This assumes your login shell is bash. If you are using a differnt [sic] shell, talk to you system admin.

export OCSSWROOT=[SeaDAS_install_dir]/ocssw
source $OCSSWROOT/OCSSW_bash.env

where: SeaDAS_install_dir is the directory where you installed SeaDAS."

Using openssl to check for the necessary ciphers doesn't ensure that python SSL is using the same library, which is why the (overly complex) python check mentioned above is useful.

aleitner · by **aleitner** » Tue Apr 11, 2017 7:14 pm America/New_York

Great! I believe that worked!
(python2) Astrids-MBP:scripts astridleitner$ python update_luts.py aqua --verbose
[ MODIS ]
+ leapsec.dat
+ utcpole.dat
[ Sensor: AQUA ]
+ OPER:MYD02_Emissive_LUTs.V6.1.35.29_OC2.hdf
+ OPER:MYD02_QA_LUTs.V6.1.35.29_OC2.hdf
+ OPER:MYD02_Reflective_LUTs.V6.1.35.29_OC2.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.17.24_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.17.27_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.31.3_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.31.5_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.31.6_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.33.1_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.33.4_OC.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.33.4_OCa.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.33.6_OC2.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.33.8_OC2.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.35.22_OC2_v2_20161207.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.35.22_OC2_v2_rvs.hdf
+ EVAL:MYD02_Emissive_LUTs.V6.1.35.25_OC2_v2_rvs3.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.17.24_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.17.27_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.31.3_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.31.5_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.31.6_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.33.1_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.33.4_OC.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.33.4_OCa.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.33.6_OC2.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.33.8_OC2.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.35.22_OC2_v2_20161207.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.35.22_OC2_v2_rvs.hdf
+ EVAL:MYD02_QA_LUTs.V6.1.35.25_OC2_v2_rvs3.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.17.24_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.17.27_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.31.3_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.31.5_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.31.6_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.33.1_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.33.4_OC.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.33.4_OCa.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.33.6_OC2.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.33.8_OC2.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.35.22_OC2_v2_20161207.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.35.22_OC2_v2_rvs.hdf
+ EVAL:MYD02_Reflective_LUTs.V6.1.35.25_OC2_v2_rvs3.hdf
+ OPER:xcal_modisa_axc39d_1240.hdf
+ OPER:xcal_modisa_axc39d_1640.hdf
+ OPER:xcal_modisa_axc39d_2130.hdf
+ OPER:xcal_modisa_axc39d_412.hdf
+ OPER:xcal_modisa_axc39d_443.hdf
+ OPER:xcal_modisa_axc39d_469.hdf
+ OPER:xcal_modisa_axc39d_488.hdf
+ OPER:xcal_modisa_axc39d_531.hdf
+ OPER:xcal_modisa_axc39d_547.hdf
+ OPER:xcal_modisa_axc39d_555.hdf
+ OPER:xcal_modisa_axc39d_645.hdf
+ OPER:xcal_modisa_axc39d_667.hdf
+ OPER:xcal_modisa_axc39d_678.hdf
+ OPER:xcal_modisa_axc39d_748.hdf
+ OPER:xcal_modisa_axc39d_859.hdf
+ OPER:xcal_modisa_axc39d_869.hdf
+ EVAL:xcal_modisa_adet01_748.hdf
+ EVAL:xcal_modisa_adet01_869.hdf
+ EVAL:xcal_modisa_axc38d_1240.hdf
+ EVAL:xcal_modisa_axc38d_1640.hdf
+ EVAL:xcal_modisa_axc38d_2130.hdf
+ EVAL:xcal_modisa_axc38d_412.hdf
+ EVAL:xcal_modisa_axc38d_443.hdf
+ EVAL:xcal_modisa_axc38d_469.hdf
+ EVAL:xcal_modisa_axc38d_488.hdf
+ EVAL:xcal_modisa_axc38d_531.hdf
+ EVAL:xcal_modisa_axc38d_547.hdf
+ EVAL:xcal_modisa_axc38d_555.hdf
+ EVAL:xcal_modisa_axc38d_645.hdf
+ EVAL:xcal_modisa_axc38d_667.hdf
+ EVAL:xcal_modisa_axc38d_678.hdf
+ EVAL:xcal_modisa_axc38d_748.hdf
+ EVAL:xcal_modisa_axc38d_859.hdf
+ EVAL:xcal_modisa_axc38d_869.hdf
+ EVAL:xcal_modisa_axc39d_1240.hdf
+ EVAL:xcal_modisa_axc39d_1640.hdf
+ EVAL:xcal_modisa_axc39d_2130.hdf
+ EVAL:xcal_modisa_axc39d_412.hdf
+ EVAL:xcal_modisa_axc39d_443.hdf
+ EVAL:xcal_modisa_axc39d_469.hdf
+ EVAL:xcal_modisa_axc39d_488.hdf
+ EVAL:xcal_modisa_axc39d_531.hdf
+ EVAL:xcal_modisa_axc39d_547.hdf
+ EVAL:xcal_modisa_axc39d_555.hdf
+ EVAL:xcal_modisa_axc39d_645.hdf
+ EVAL:xcal_modisa_axc39d_667.hdf
+ EVAL:xcal_modisa_axc39d_678.hdf
+ EVAL:xcal_modisa_axc39d_748.hdf
+ EVAL:xcal_modisa_axc39d_859.hdf
+ EVAL:xcal_modisa_axc39d_869.hdf
+ EVAL:xcal_modisa_axc40d_1240.hdf
+ EVAL:xcal_modisa_axc40d_1640.hdf
+ EVAL:xcal_modisa_axc40d_2130.hdf
+ EVAL:xcal_modisa_axc40d_412.hdf
+ EVAL:xcal_modisa_axc40d_443.hdf
+ EVAL:xcal_modisa_axc40d_469.hdf
+ EVAL:xcal_modisa_axc40d_488.hdf
+ EVAL:xcal_modisa_axc40d_531.hdf
+ EVAL:xcal_modisa_axc40d_547.hdf
+ EVAL:xcal_modisa_axc40d_555.hdf
+ EVAL:xcal_modisa_axc40d_645.hdf
+ EVAL:xcal_modisa_axc40d_667.hdf
+ EVAL:xcal_modisa_axc40d_678.hdf
+ EVAL:xcal_modisa_axc40d_748.hdf
+ EVAL:xcal_modisa_axc40d_859.hdf
+ EVAL:xcal_modisa_axc40d_869.hdf
+ EVAL:xcal_modisa_axc41d_1240.hdf
+ EVAL:xcal_modisa_axc41d_1640.hdf
+ EVAL:xcal_modisa_axc41d_2130.hdf
+ EVAL:xcal_modisa_axc41d_412.hdf
+ EVAL:xcal_modisa_axc41d_443.hdf
+ EVAL:xcal_modisa_axc41d_469.hdf
+ EVAL:xcal_modisa_axc41d_488.hdf
+ EVAL:xcal_modisa_axc41d_531.hdf
+ EVAL:xcal_modisa_axc41d_547.hdf
+ EVAL:xcal_modisa_axc41d_555.hdf
+ EVAL:xcal_modisa_axc41d_645.hdf
+ EVAL:xcal_modisa_axc41d_667.hdf
+ EVAL:xcal_modisa_axc41d_678.hdf
+ EVAL:xcal_modisa_axc41d_748.hdf
+ EVAL:xcal_modisa_axc41d_859.hdf
+ EVAL:xcal_modisa_axc41d_869.hdf
+ EVAL:xcal_modisa_axc42d_1240.hdf
+ EVAL:xcal_modisa_axc42d_1640.hdf
+ EVAL:xcal_modisa_axc42d_2130.hdf
+ EVAL:xcal_modisa_axc42d_412.hdf
+ EVAL:xcal_modisa_axc42d_443.hdf
+ EVAL:xcal_modisa_axc42d_469.hdf
+ EVAL:xcal_modisa_axc42d_488.hdf
+ EVAL:xcal_modisa_axc42d_531.hdf
+ EVAL:xcal_modisa_axc42d_547.hdf
+ EVAL:xcal_modisa_axc42d_555.hdf
+ EVAL:xcal_modisa_axc42d_645.hdf
+ EVAL:xcal_modisa_axc42d_667.hdf
+ EVAL:xcal_modisa_axc42d_678.hdf
+ EVAL:xcal_modisa_axc42d_748.hdf
+ EVAL:xcal_modisa_axc42d_859.hdf
+ EVAL:xcal_modisa_axc42d_869.hdf
[ Done ]

Is there any way to test and make sure everything is working as it should be?

gnwiii · by **gnwiii** » Tue Apr 11, 2017 7:49 pm America/New_York

Good to see things getting back on track. I'm sure there are others who can benefit from seeing how to have both Anaconda Python2 and Python3 installed.

Does update_luts.py work in the GUI as wells on the command line?

To check that the processing system is working, you can start with the (somewhat outdated) benchmark. This provides a bash script to process a level-0 file MODIS Aqua file up to an SMI image. It can be a useful exercise to figure out how to make this work with the current OCSSW software to produce netcdf4-cf output.

aleitner · by **aleitner** » Tue Apr 11, 2017 8:45 pm America/New_York

The update processors still cannot make it past the python version check for me, so I don't know if it would work through the GUI. I will try the benchmark check.
Thank you so much!

by **OB SeaDAS - knowles** » Wed Apr 12, 2017 7:04 am America/New_York

You've probably seen this FAQ, but just to make sure people see it in this thread:
https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=6542

The 2nd part of the FAQ is important about deleting the OCSSW directory and starting over. Also you may need to reboot your computer in order to make sure seadas is being launched in the environment with your correct python version. Installing from the GUI is easy and should (hopefully) work for you.

Danny