SeaDAS 8 install_OCSSW problem with python requests

Please enter here to ask a question about any NASA Science related topics!
daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Mar 19, 2021 10:39 am America/New_York

I've just installed SeaDAS 8.0.0 Beta with the shell script for Mac (Catalina 10.15.7) without incident. I then tried to install OCSSW on the command line and it failed to install LUTs because python was unable to import requests:

(base) daurin@GSSLA18091785:~/Downloads$ ./install_ocssw --list_tags
R2020.1
T2020.0
T2020.1
T2020.2
T2020.3
T2020.4
T2020.5
T2020.6
T2020.7
T2021.0
T2021.3
T2021.4
T2021.5
V2020.0
V2020.1
V2020.2
V2021.0
V2021.1
don
initial
v3.1.0
v3.1.0-rc
v3.1.0-rc1
(base) daurin@GSSLA18091785:~/Downloads$ ./install_ocssw --install_dir ~/Applications/seadas8/ocssw --tag V2021.1 --seadas --modisa --seawifs --modist --msis2a --msis2b --olcis3a --olcis3b --viirsj1 --viirsn
Installing (1 of 19) - bin_macosx_intel
Installing (2 of 19) - lib_macosx_intel
Installing (3 of 19) - opt_macosx_intel
Installing (4 of 19) - root
Installing (5 of 19) - common
Installing (6 of 19) - modis
Installing (7 of 19) - modisa
Installing (8 of 19) - modist
Installing (9 of 19) - msi
Installing (10 of 19) - msis2a
Installing (11 of 19) - msis2b
Installing (12 of 19) - ocrvc
Installing (13 of 19) - olci
Installing (14 of 19) - olcis3a
Installing (15 of 19) - olcis3b
Installing (16 of 19) - seawifs
Installing (17 of 19) - viirs
Installing (18 of 19) - viirsj1
Installing (19 of 19) - viirsn
Installing lut - modisa
Traceback (most recent call last):
File "/Users/daurin/Applications/seadas8/ocssw/bin/update_luts", line 9, in <module>
import seadasutils.LutUtils as Lut
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/LutUtils.py", line 3, in <module>
import seadasutils.JsonUtils as Session
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/JsonUtils.py", line 6, in <module>
from seadasutils.ProcUtils import getSession, httpdl
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/ProcUtils.py", line 13, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
Error: return = 1 : trying to run command = /Users/daurin/Applications/seadas8/ocssw/bin/ocssw_runner --ocsswroot /Users/daurin/Applications/seadas8/ocssw update_luts modisa


I checked that my Anaconda python had requests:

(base) daurin@GSSLA18091785:~/Downloads$ python --version
Python 3.8.3
(base) daurin@GSSLA18091785:~/Downloads$ python
Python 3.8.3 (default, Jul 2 2020, 11:26:31)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> exit()
(base) daurin@GSSLA18091785:~/Downloads$


I then tried to install OCSSW through the SeaDAS GUI with the same result:

execution exception: java.io.IOException: install_ocssw failed with exit code 1.
Check log for more details.
Installing (1 of 13) - bin_macosx_intel
Installing (2 of 13) - lib_macosx_intel
Installing (3 of 13) - opt_macosx_intel
Installing (4 of 13) - root
Installing (5 of 13) - common
Installing (6 of 13) - modis
Installing (7 of 13) - modisa
Installing (8 of 13) - modist
Installing (9 of 13) - ocrvc
Installing (10 of 13) - seawifs
Installing (11 of 13) - viirs
Installing (12 of 13) - viirsj1
Installing (13 of 13) - viirsn
Traceback (most recent call last):
File "/Users/daurin/Applications/seadas8/ocssw/bin/update_luts", line 9, in <module>
import seadasutils.LutUtils as Lut
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/LutUtils.py", line 3, in <module>
import seadasutils.JsonUtils as Session
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/JsonUtils.py", line 6, in <module>
from seadasutils.ProcUtils import getSession, httpdl
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/ProcUtils.py", line 13, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
Installing lut - modisa
Error: return = 1 : trying to run command = /Users/daurin/Applications/seadas8/ocssw/bin/ocssw_runner --ocsswroot /Users/daurin/Applications/seadas8/ocssw update_luts modisa


Looks like install_ocssw is pulling up my old Python2 installation (kept around for testing reverse compatibility in development):

(base) daurin@GSSLA18091785:~/Downloads$ python2

WARNING: Python 2.7 is not recommended.
This version is included in macOS for compatibility with legacy software.
Future versions of macOS will not include Python 2.7.
Instead, it is recommended that you transition to using 'python3' from within Terminal.

Python 2.7.16 (default, Jun 5 2020, 22:59:21)
[GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.29.20) (-macos10.15-objc- on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
>>>


A related Ocean Color Forum post: https://oceancolor.gsfc.nasa.gov/forum/ ... ?tid=12979
How can this be avoided when running install_ocssw without changing my python2 installation (i.e. how do I force install_ocssw to run in python3)? Thanks!

Dirk

Tags:

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 220
Joined: Thu Mar 05, 2009 10:25 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by OB SeaDAS - dshea » Fri Mar 19, 2021 12:03 pm America/New_York

The update_luts script uses python3. Can you run a few commands for me and send me the output?

echo $PATH
which update_luts
which python3


don

gnwiii
Posts: 642
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

Re: SeaDAS 8 install_OCSSW problem with python requests

by gnwiii » Fri Mar 19, 2021 1:21 pm America/New_York

With previous OCSSW versions it was not unusual to see failures in update_luts.py, but since that is the last step, the installations were essentially complete and running update_luts could be done without reinstalling. I don't currently have access to macOS, but for linux, "head -1 ~/ocssw/bin/update_luts" gives "#! /usr/bin/env python3". When I had macOS (El Capitan, so a while back, and using python2) I put a symbolic link to the python program I wanted to use "$OCSSWROOT/opt/bin" to ensure the scripts used the right "python".

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Mar 19, 2021 1:26 pm America/New_York

Hi Don,

echo $PATH
/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/accounts/daurin/anaconda3/bin:/Applications/seadas-7.5.3/ocssw/opt/bin:/Applications/seadas-7.5.3/ocssw/scripts:/Applications/seadas-7.5.3/ocssw/bin:/accounts/daurin/Matlab/2018b/bin:/anaconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin
[Looks like I should update my .bashrc for the new SeaDAS install]

which update_luts
[none/no reply]

which python3
/Users/daurin/anaconda3/bin/python3

Dirk

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Mar 19, 2021 1:40 pm America/New_York

George,

First line of the header in my bin/update_luts in the new installation directory for seadas8/ocssw is the same: #! /usr/bin/env python3.

Didn't quite follow that about the symbolic link. Did you mean place a link to my python3 within $OCSSWROOT/opt/bin?

Thanks,
Dirk

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 220
Joined: Thu Mar 05, 2009 10:25 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by OB SeaDAS - dshea » Fri Mar 19, 2021 1:51 pm America/New_York

Once your .bashrc points to the new seadas8 stuff you should be fine. I am pretty sure that the OCSSW from seadas7.5.3 had troubles with python3.

don

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Mar 19, 2021 1:53 pm America/New_York

I updated my bashrc to point to the new ocssw:
# export OCSSWROOT="/Applications/seadas-7.5.3/ocssw"
export OCSSWROOT="/Users/daurin/Applications/seadas8/ocssw"
source $OCSSWROOT/OCSSW_bash.env

Sourced the bashrc:
(base) daurin@GSSLA18091785:~$ echo $PATH
/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/accounts/daurin/anaconda3/bin:/Users/daurin/Applications/seadas8/ocssw/opt/bin:/Users/daurin/Applications/seadas8/ocssw/bin:/accounts/daurin/Matlab/2018b/bin:/anaconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin

But still get the same result: No module named 'requests'.

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Mar 19, 2021 4:03 pm America/New_York

I went through my bashrc and bash_profile to clean up some garbage ported over from my linux machine. My PATH is now clean:

/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/Users/daurin/Applications/seadas8/ocssw/opt/bin:/Users/daurin/Applications/seadas8/ocssw/bin:/Applications/MATLAB_R2020a.app/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin

But install_ocssw is still bombing with the same error:
execution exception: java.io.IOException: install_ocssw failed with exit code 1.
Check log for more details.
Traceback (most recent call last):
File "/var/folders/th/7q7vkd_x68g_88b8j5qg5kxh0000gp/T/install_ocssw", line 18, in <module>
import manifest as mf
File "/private/var/folders/th/7q7vkd_x68g_88b8j5qg5kxh0000gp/T/manifest.py", line 16, in <module>
import requests
ModuleNotFoundError: No module named 'requests'

I then created a symbolic link to /Users/daurin/anaconda3/bin/python in /Users/daurin/Applications/seadas8/ocssw/opt/bin, but got the same result.

gnwiii
Posts: 642
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

Re: SeaDAS 8 install_OCSSW problem with python requests

by gnwiii » Fri Mar 19, 2021 5:26 pm America/New_York

Anaconda installs on Windows and linux don't include a python3 executable, so you may only have "$HOME/anacoda3/bin/python".
In a terminal (in a terminal, after checking that the PATH is correct, what does "which python3" give? If it gives "/usr/bin/python3" you are still using Apple's system python3 which should be reserved for Apple's use. Your symbolic link should have been created using "ln -s $HOME/anacoda3/bin/python" "$HOME/Applications/seadas8/ocssw/opt/bin/python3". You can check that the requests module has been installed in your Anaconda3 by running:
"/Users/daurin/anaconda3/bin/python -c 'import requests'". If it is not found, use the package manager to install it.

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Mar 19, 2021 6:07 pm America/New_York

Okay, thanks. That was what I had done.

(base) daurin@GSSLA18091785:~/Applications/seadas8/ocssw/opt/bin$ which python3
/Users/daurin/anaconda3/bin/python3

Here's the link:
(base) daurin@GSSLA18091785:~/Applications/seadas8/ocssw/opt/bin$ ll python3
lrwxr-xr-x 1 daurin staff 35 Mar 19 16:02 python3@ -> /Users/daurin/anaconda3/bin/python3

No problem importing requests when I run python, python3, ~/anaconda3/bin/python, ~/anaconda3/bin/python3, nor ~/Applications/seadas8/opt/bin/python3, only if I run python2 and install_ocssw is still throwing ModuleNotFoundError when it tries to import requests.

Post Reply