Earthdata Forum

I'm trying a clean install of ocssw and am having similar problems.
I have tried with python 2.7.5 and 2.7.11. (both higher than 2.6.5 or above and I believe that my last install of 7.4 used python 2.7.5)

Installing ocrvc (5 of 33)
Installing aquarius (6 of 33)
Installing avhrr (7 of 33)
Installing czcs (8 of 33)
Installing goci (9 of 33)
Installing hico (10 of 33)
Installing meris (11 of 33)
Installing modis (12 of 33)
Installing modisa (13 of 33)
Installing hmodisa (14 of 33)
Installing modist (15 of 33)
Installing hmodist (16 of 33)
Installing mos (17 of 33)
Installing ocm1 (18 of 33)
Installing ocm2 (19 of 33)
Installing octs (20 of 33)
Installing osmi (21 of 33)
Installing oli (22 of 33)
Installing seawifs (23 of 33)
Installing viirsn (24 of 33)
Installing bin (25 of 33)
Installing bin3 (26 of 33)
Installing scripts (27 of 33)
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "/shares/cms_optics/apps/seadas/seadas-7.4/ocssw/run/scripts/pyverchk.py", line 3, in <module>
    import re
  File "/apps/python/2.7.11/lib/python2.7/re.py", line 105, in <module>
    import sre_compile
  File "/apps/python/2.7.11/lib/python2.7/sre_compile.py", line 62
    for t in _equivalences for i in t}
      ^
SyntaxError: invalid syntax
Error - Python version is not new enough to install luts

HOWEVER, when I use 2.6.6 it seems to work fine? Why is this?

Thanks,
B

I don't see the similarity.   This looks like a problem with your python 2.7.11 installation -- perhaps mixing 32-bit and 64-bit code.  You should mention the platform and the source of your python installation.   I'd say just go with the python 2.6 version, but it is important to check the ssl support in python so update_luts.py can use https URL's.  The python version is not enough to ensure that https will work, as https support also depends on the version of the ssl library used by python.   SSL support was improved in python 2.7, but some Red Hat systems have usable ssl support in python 2.6.

Gnwii,

Thanks, but I don't think that is it. The binaries seem to be 64bit and see below that update_luts.py seems to works with 2.6.6; 2.7.5 has [50993 refs]
included for some reason (see bolded below); and 2.7.11 seems to work. However, the 2.7.5 and 2.7.11 failed from the seadas gui to update the occsw code with the errors I submitted above when I have the 2.7.5 or 2.7.11 python environment. The systems admin on our cluster is pretty great at making various version of python available. Here is the error with 2.7.5 (which is not above):

Installing ocrvc (5 of 13)
Installing modis (6 of 13)
Installing modisa (7 of 13)
Installing hmodisa (8 of 13)
Installing bin (9 of 13)
Installing bin3 (10 of 13)
Installing scripts (11 of 13)
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "/shares/cms_optics/apps/seadas/seadas-7.4/ocssw/run/scripts/pyverchk.py", line 3, in <module>
    import re
  File "/apps/python/2.7.5/lib/python2.7/re.py", line 105, in <module>
    import sre_compile
  File "/apps/python/2.7.5/lib/python2.7/sre_compile.py", line 14, in <module>
    import sre_parse
  File "/apps/python/2.7.5/lib/python2.7/sre_parse.py", line 17, in <module>
    from sre_constants import *
  File "/apps/python/2.7.5/lib/python2.7/sre_constants.py", line 18, in <module>
    from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT
Error - Python version is not new enough to install luts
[45153 refs]

[bmurch@optics0 data]$ module avail apps/python

--------------------------------------------------------------------------------------------------- /etc/modulefiles ----------------------------------------------------------------------------------------------------
apps/python/2.7.11       apps/python/2.7.5        apps/python/3.4.3        apps/python/3.5.1        apps/python/3.6.2        apps/python/intelpython2 apps/python/intelpython3

With regards to the cluster platform, all clustered machines are Redhat supported:
[bmurch@optics0 seadas-7.4]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.8 (Santiago)

See:
[bmurch@optics0 seadas-7.4]$ module purge
[bmurch@optics0 seadas-7.4]$ which python
/usr/bin/python
[bmurch@optics0 seadas-7.4]$ python -V
Python 2.6.6
[bmurch@optics0 seadas-7.4]$ file /usr/bin/python
/usr/bin/python: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped
[bmurch@optics0 seadas-7.4]$ python $OCSSWROOT/run/scripts/update_luts.py aqua --verbose
[ MODIS ]
+ leapsec.dat
+ utcpole.dat
[ Sensor: AQUA ]
[ Done ]

[bmurch@optics0 seadas-7.4]$ module load apps/python/2.7.5
[bmurch@optics0 seadas-7.4]$ python -V
Python 2.7.5
[bmurch@optics0 seadas-7.4]$ which python
/apps/python/2.7.5/bin/python
[bmurch@optics0 seadas-7.4]$ file /apps/python/2.7.5/bin/python
/apps/python/2.7.5/bin/python: symbolic link to `python2'
[bmurch@optics0 seadas-7.4]$ file /apps/python/2.7.5/bin/python2
/apps/python/2.7.5/bin/python2: symbolic link to `python2.7'
[bmurch@optics0 seadas-7.4]$ file /apps/python/2.7.5/bin/python2.7
/apps/python/2.7.5/bin/python2.7: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, not stripped
[bmurch@optics0 seadas-7.4]$ python $OCSSWROOT/run/scripts/update_luts.py aqua --verbose
[ MODIS ]
+ leapsec.dat
+ utcpole.dat
[ Sensor: AQUA ]
[ Done ]

[50993 refs]
[bmurch@optics0 seadas-7.4]$ module purge
[bmurch@optics0 seadas-7.4]$ which python
/usr/bin/python
[bmurch@optics0 seadas-7.4]$ module load apps/python/2.7.11
[bmurch@optics0 seadas-7.4]$ which python
/apps/python/2.7.11/bin/python
[bmurch@optics0 seadas-7.4]$ python -V
Python 2.7.11
[bmurch@optics0 seadas-7.4]$ file /apps/python/2.7.11/bin/python
/apps/python/2.7.11/bin/python: symbolic link to `python2'
[bmurch@optics0 seadas-7.4]$ file /apps/python/2.7.11/bin/python2
/apps/python/2.7.11/bin/python2: symbolic link to `python2.7'
[bmurch@optics0 seadas-7.4]$ file /apps/python/2.7.11/bin/python2.7
/apps/python/2.7.11/bin/python2.7: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, not stripped
[bmurch@optics0 seadas-7.4]$ python $OCSSWROOT/run/scripts/update_luts.py aqua --verbose
[ MODIS ]
+ leapsec.dat
+ utcpole.dat
[ Sensor: AQUA ]
[ Done ]

The OCSSW python scripts are pretty basic python and should not depend on a particular version until you start messing with https URL's.  This suggests that your problems are with the local python installation.   There are internal changes from one version to the next, and the "ImportError: cannot import name MAXREPEAT" message looks like a version mismatch between the interpreter and the python import libraries.  I see you are using environment modules -- do the modules adjust the PYTHONPATH?  It might be informative to compare sys.path across interpreters.

Have you considered using Red Hat's scl python 2.7.13?

Thanks, but I doubt that I would be able to install much of anything in terms of our computing cluster. But here's the thing gnwii, when I run update_luts.py from the command line, they work fine in all three versions! See below. It's only through the GUI that 2.7.5 and 2.7.11 fail.

Here are the contents of sys.path with different python modules and I don't see any cross contamination. I have a couple of scripts in bold below:

[oo@optics0 python3]$ cat path_test.py
import sys
print(sys.path)

[oo@optics0 python3]$ cat /shares/cms_optics/apps/seadas/cron/ocssw_7.4-CentOS6.sh
#!/bin/bash                                                                          
#$1 should be satellite                 
. /shares/cms_optics/apps/seadas/seadas-7.4/ocssw/OCSSW_bash_module.env              
command="python $OCSSWROOT/run/scripts/update_luts.py $1 --verbose"                  
eval $command 

[oo@optics0 python3]$ which python                                                                                                                                                                           
/usr/bin/python
[oo@optics0 python3]$ python -V
Python 2.6.6
[oo@optics0 python3]$ python path_test.py
['/home/o/oo/programming/python3', '/usr/lib64/python26.zip', '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old', '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages', '/usr/lib64/python2.6/site-packages/Numeric', '/usr/lib64/python2.6/site-packages/PIL', '/usr/lib64/python2.6/site-packages/gtk-2.0', '/usr/lib/python2.6/site-packages', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info']
[oo@optics0 python3]$ /shares/cms_optics/apps/seadas/cron/ocssw_7.4-CentOS6.sh aqua
[ MODIS ]                                                                             
+ leapsec.dat                                                                         
+ utcpole.dat                                                                         
[ Sensor: AQUA ]                                                                      
[ Done ]                                                                              

[oo@optics0 python3]$ module load apps/python/2.7.5
[oo@optics0 python3]$ which python
/apps/python/2.7.5/bin/python
[oo@optics0 python3]$ python -V
Python 2.7.5
[oo@optics0 python3]$ python path_test.py
['/home/o/oo/programming/python3', '/apps/python/2.7.5/lib/python2.7/site-packages/gemini-0.18.0-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/numexpr-2.4.6-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/bcolz-0.12.1-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/cyordereddict-0.2.2-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/Unidecode-0.4.18-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/openpyxl-1.8.6-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/pandas-0.17.1-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/bx_python-0.7.3-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/ipython_cluster_helper-0.5.0-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/ipyparallel-4.1.0-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/bottle-0.12.9-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/networkx-1.10-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/pybedtools-0.7.4-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/pysam-0.8.4-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/geneimpacts-0.0.5-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/inheritance-0.0.4-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/pytz-2015.7-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/netifaces-0.10.4-py2.7-linux-x86_64.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/ipython-4.0.1-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/pexpect-4.0.1-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/simplegeneric-0.8.1-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/pickleshare-0.5-py2.7.egg', '/apps/python/2.7.5/lib/python2.7/site-packages/path.py-8.1.2-py2.7.egg', '/apps/python/2.7.5/lib/python2.7', '/apps/python/2.7.5/packages/lib/python2.7/site-packages', '/apps/python/2.7.5/lib/python', '/apps/python/2.7.5/lib/python27.zip', '/apps/python/2.7.5/lib/python2.7/plat-linux2', '/apps/python/2.7.5/lib/python2.7/lib-tk', '/apps/python/2.7.5/lib/python2.7/lib-old', '/apps/python/2.7.5/lib/python2.7/lib-dynload', '/apps/python/2.7.5/lib/python2.7/site-packages', '/apps/python/2.7.5/lib/python2.7/site-packages/PIL', '/apps/python/2.7.5/lib/python2.7/site-packages/setuptools-18.8-py2.7.egg']
[33464 refs]
[oo@optics0 python3]$ /shares/cms_optics/apps/seadas/cron/ocssw_7.4-CentOS6.sh aqua
[ MODIS ]                                                                             
+ leapsec.dat                                                                         
+ utcpole.dat                                                                         
[ Sensor: AQUA ]
[ Done ]

[50993 refs]
[oo@optics0 python3]$ module unload apps/python/2.7.5
[oo@optics0 python3]$ python -V
Python 2.6.6
[oo@optics0 python3]$ module load apps/python/2.7.11
[oo@optics0 python3]$ python -V
Python 2.7.11
[oo@optics0 python3]$ python path_test.py
['/home/o/oo/programming/python3', '/apps/python/2.7.11/lib/python2.7', '/apps/python/2.7.11/lib/python2.7/site-packages', '/apps/python/2.7.11/lib/python27.zip', '/apps/python/2.7.11/lib/python2.7/plat-linux2', '/apps/python/2.7.11/lib/python2.7/lib-tk', '/apps/python/2.7.11/lib/python2.7/lib-old', '/apps/python/2.7.11/lib/python2.7/lib-dynload', '/apps/python/2.7.11/lib/python2.7/site-packages/gdc_client-1.2.0-py2.7.egg', '/apps/python/2.7.11/lib/python2.7/site-packages/setuptools-19.2-py2.7.egg', '/apps/python/2.7.11/lib/python2.7/site-packages/PyYAML-3.11-py2.7-linux-x86_64.egg', '/apps/python/2.7.11/lib/python2.7/site-packages/lxml-3.5.0b1-py2.7-linux-x86_64.egg', '/apps/python/2.7.11/lib/python2.7/site-packages/parcel-0.1.13-py2.7.egg']
[oo@optics0 python3]$ /shares/cms_optics/apps/seadas/cron/ocssw_7.4-CentOS6.sh aqua
[ MODIS ]
+ leapsec.dat
+ utcpole.dat
[ Sensor: AQUA ]
[ Done ]

All the paths have '/home/o/oo/programming/python3' which could be a problem if that directory contains a module with the same same as something in the /apps/python/2.7.NN tree.  The difference between GUI and command-line is a hint.  The GUI uses $OCSSWROOT/run/scripts/ocssw_runner which sources $OCSSWROOT/OCSSW_bash.env, so may conflict with whatever special sauce  OCSSW_bash_module.env provides.  For python 2.7.5 the [33464 refs] message (which might indicate that your python 2.7.5 is a debugging build) could be confusing something in the GUI trying to parse output from python scripts, but doesn't explain why 2.7.11 fails.

You could get some more detailes by tweaking update_luts.py to record things like sys.path to a file (as using stdout might pollute messages the GUI uses).