SeaDAS 8 install_OCSSW problem with python requests

Please enter here to ask a question about any NASA Science related topics!
daurin
Posts: 21
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Thu Mar 25, 2021 10:09 am America/New_York

Here is the whole thing again in a fresh shell:

(base) daurin@GSSLA18091785:~$ echo $PATH
/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/Users/daurin/Applications/seadas8/ocssw/opt/bin:/Users/daurin/Applications/seadas8/ocssw/bin:/Applications/MATLAB_R2020a.app/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin
(base) daurin@GSSLA18091785:~$ which python3
/Users/daurin/anaconda3/bin/python3
(base) daurin@GSSLA18091785:~$ which python
/Users/daurin/anaconda3/bin/python
(base) daurin@GSSLA18091785:~$ which python2
/usr/bin/python2
(base) daurin@GSSLA18091785:~$ python3
Python 3.8.3 (default, Jul 2 2020, 11:26:31)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> exit()
(base) daurin@GSSLA18091785:~$ cd ~/Downloads/
(base) daurin@GSSLA18091785:~/Downloads$ ./install_ocssw --install_dir ~/Applications/seadas8/ocssw --tag V2021.1 --seadas --modisa --seawifs --modist --msis2a --msis2b --olcis3a --olcis3b --viirsj1 --viirsn
Installing (1 of 19) - bin_macosx_intel
Installing (2 of 19) - lib_macosx_intel
Installing (3 of 19) - opt_macosx_intel
Installing (4 of 19) - root
Installing (5 of 19) - common
Installing (6 of 19) - modis
Installing (7 of 19) - modisa
Installing (8 of 19) - modist
Installing (9 of 19) - msi
Installing (10 of 19) - msis2a
Installing (11 of 19) - msis2b
Installing (12 of 19) - ocrvc
Installing (13 of 19) - olci
Installing (14 of 19) - olcis3a
Installing (15 of 19) - olcis3b
Installing (16 of 19) - seawifs
Installing (17 of 19) - viirs
Installing (18 of 19) - viirsj1
Installing (19 of 19) - viirsn
Installing lut - modisa
Traceback (most recent call last):
File "/Users/daurin/Applications/seadas8/ocssw/bin/update_luts", line 9, in <module>
import seadasutils.LutUtils as Lut
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/LutUtils.py", line 3, in <module>
import seadasutils.JsonUtils as Session
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/JsonUtils.py", line 6, in <module>
from seadasutils.ProcUtils import getSession, httpdl
File "/Users/daurin/Applications/seadas8/ocssw/bin/seadasutils/ProcUtils.py", line 13, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
Error: return = 1 : trying to run command = /Users/daurin/Applications/seadas8/ocssw/bin/ocssw_runner --ocsswroot /Users/daurin/Applications/seadas8/ocssw update_luts modisa

(base) daurin@GSSLA18091785:~/Downloads$ ocssw_runner --ocsswroot $OCSSWROOT env
LIB3_DIR=/Users/daurin/Applications/seadas8/ocssw/opt
PROJ_LIB=/Users/daurin/Applications/seadas8/ocssw/opt/share/proj
LIB3_CHECK=0
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/usr/local/bin/bash
LIB3_LIB=/Users/daurin/Applications/seadas8/ocssw/opt/lib
TMPDIR=/var/folders/th/7q7vkd_x68g_88b8j5qg5kxh0000gp/T/
CONDA_SHLVL=1
CONDA_PROMPT_MODIFIER=(base)
TERM_PROGRAM_VERSION=433
TERM_SESSION_ID=3829E37E-4F73-4A24-8052-B9280A852AED
LIB3_BIN=/Users/daurin/Applications/seadas8/ocssw/opt/bin
LIB3_INC=/Users/daurin/Applications/seadas8/ocssw/opt/include
USER=daurin
SWTBL=/Users/daurin/Applications/seadas8/ocssw/share/seawifs
CONDA_EXE=/Users/daurin/anaconda3/bin/conda
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.h63i0khAMj/Listeners
OCVARROOT=/Users/daurin/Applications/seadas8/ocssw/var
OCSSW_ARCH=macosx_intel
_CE_CONDA=
ASAP_PARMS=/Users/daurin/Applications/seadas8/ocssw/share/common/nav
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/Users/daurin/Applications/seadas8/ocssw/opt/bin:/Users/daurin/Applications/seadas8/ocssw/bin:/Applications/MATLAB_R2020a.app/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin
L2GEN_ANC=/Users/daurin/Applications/seadas8/ocssw/var/anc
NAVCTL=/Users/daurin/Applications/seadas8/ocssw/share/seawifs/nav
OCSSWROOT=/Users/daurin/Applications/seadas8/ocssw
CONDA_PREFIX=/Users/daurin/anaconda3
HDFEOS_LIB=/Users/daurin/Applications/seadas8/ocssw/opt/EOS/lib/macintel64
OCSSW_BIN=/Users/daurin/Applications/seadas8/ocssw/bin
HRPT_STATION_IDENTIFICATION_FILE=/Users/daurin/Applications/seadas8/ocssw/share/seawifs/station/station_id.dat
PWD=/Users/daurin/Downloads
LANG=en_US.UTF-8
OCSSW_SVN=localhost
XPC_FLAGS=0x0
OCDATAROOT=/Users/daurin/Applications/seadas8/ocssw/share
SWFTBL=/Users/daurin/Applications/seadas8/ocssw/share/seawifs/qc
XPC_SERVICE_NAME=0
OCSSW_DEBUG=0
PGSLIB=/Users/daurin/Applications/seadas8/ocssw/opt/EOS/lib/macintel64
CXX=g++
_CE_M=
SHLVL=2
HOME=/Users/daurin
FC=gfortran
CONDA_PYTHON_EXE=/Users/daurin/anaconda3/bin/python
LOGNAME=daurin
ORBCTL=/Users/daurin/Applications/seadas8/ocssw/share/seawifs/nav
GCC_TUNE=
CONDA_DEFAULT_ENV=base
OCTS_REGISTRATION_TABLES=/Users/daurin/Applications/seadas8/ocssw/share/octs/nav
CC=gcc
DISPLAY=/private/tmp/com.apple.launchd.LsyV81Pece/org.xquartz:0
EOS_LIB_PREFIX=macintel64
ELEMENTS=/Users/daurin/Applications/seadas8/ocssw/var/seawifs
PGSINC=/Users/daurin/Applications/seadas8/ocssw/opt/EOS/include
NAVQC=/Users/daurin/Applications/seadas8/ocssw/share/seawifs/nav
_=/usr/bin/env
(base) daurin@GSSLA18091785:~/Downloads$ echo $PATH
/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/Users/daurin/Applications/seadas8/ocssw/opt/bin:/Users/daurin/Applications/seadas8/ocssw/bin:/Applications/MATLAB_R2020a.app/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin
(base) daurin@GSSLA18091785:~/Downloads$

This thread has exploded, so I'm going to work with Don offline to trace the problem, and we'll post the results here. Thanks to you both.

Dirk

Tags:

daurin
Posts: 21
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Fri Apr 02, 2021 9:21 am America/New_York

To follow up, Don and I were unable to trace the problem, but we did create a workaround.

With my .bashrc and .bash_profile set up as they should be, update_luts (when called by install_ocssw) was and is calling a version of python that did not have the requests module.

There are three pythons on my computer: macOS' default python2 (usr/bin/python2, v2.7.16), macOS' default python3 (/usr/bin/python3, v3.8.2), and anaconda's python3 (/User/daurin/anaconda3/bin/python3.8 aka python3 aka python.app aka python, v3.8.3). Both versions of python3 have requests, but python2 does not. Neither of us have any idea how update_luts was calling python2, but it appeared to be.

What worked was adding a line in .bashrc to force anaconda3/bin to the front of the path (again) before setting up the OCSSW environment. Anaconda already has a code block in .bash_profile that puts anaconda/bin before /usr/anything, but only by doing it AGAIN in the .bashrc does install_ocssw work properly. So, in the .bashrc, add:

export PATH="/Users/daurin/anaconda3/bin:$PATH"
before the usual SeaDAS stuff:
export OCSSWROOT="/Users/daurin/Applications/seadas8/ocssw"
source $OCSSWROOT/OCSSW_bash.env

Now, my path looks like this (note the redundant listing for /Users/daurin/anaconda3/bin):

/Users/daurin/Applications/seadas8/ocssw/opt/bin:/Users/daurin/Applications/seadas8/ocssw/bin:/Users/daurin/anaconda3/bin:/Applications/MATLAB_R2020a.app/bin:/Users/daurin/anaconda3/bin:/Users/daurin/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Visual Studio Code.app/Contents/Resources/app/bin

We cannot explain why this works, but I, for one, am moving on for now.

gnwiii
Posts: 606
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 1

Re: SeaDAS 8 install_OCSSW problem with python requests

by gnwiii » Fri Apr 02, 2021 3:06 pm America/New_York

I'm glad you got it working. There are, however, other people having problems with python for ESA SNAP snappy on macOS, and I expect SeaDAS 8 users will also get snappy with the final release, so it would be nice to understand how macOS manages the PATH. It seems to me that the default path is somehow being added after "~/.bashrc" is run at startup. I will pursue this in Apple Developers forums.

I recall that you also had a "/usr/local/bin/python" (have you installed python with homebrew?).

daurin
Posts: 21
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: SeaDAS 8 install_OCSSW problem with python requests

by daurin » Mon Apr 05, 2021 1:34 pm America/New_York

Yes, you are right! There is yet /another/ python3 on this machine I was forgetting. Origin unknown. The links in /usr/local/bin ("python3" and "python3.9") point to usr/local/Cellar/python@3.9/3.9.1_1/Frameworks/Python.framework/Versions/3.9/bin/python3.9.

And here is the thing: if I run THAT python3, it does not have the requests module; I get the ModuleNotFoundError that started this thread! I think you're onto something.

Post Reply