L1A to modis_geo to L1B problem

[davouris]

Hello! I am trying to generate L2 MODIS files from the L1A_LAC files downloaded from the OC browser. I understand I need to first create the MODIS_GEO file before I run the MODISL1B processor. This is the error I get when I run the MODIS_GEO processor. I am using the GUI...

execution exception: java.io.IOException: modis_GEO.py failed with exit code 1.
Check log for more details.
Error! could not establish a network connection. Check your network connection.
If you do not find a problem, please try again later.
Determining required attitude and ephemeris files...

Input file: /Users/dulci/Documents/MODIS/LakeErie2011/A2011246185500.L1A_LAC
Start time: 2011246185500
End time: 2011246185959

Cannot create geolocation from /Users/dulci/Documents/MODIS/LakeErie2011/A2011246185500.L1A_LAC; exiting.

Any thoughts? Or please direct me to where this issue has already been discussed?
-dulci

[OB SeaDAS - dshea]

You need to update your OCSSW install manually.  The SeaDAS GUI will updated soon to fix the problem.  NASA was required to move from http to https which caused a bunch of issues.  Here are the direction to follow to update your OCSSW programs.

https://seadas.gsfc.nasa.gov/downloads/#manual-installation

don

[davouris]

Hi Don,

Thanks for the reply. I had just fought my way through the newest OCSSW update manually. The failure of the MODIS_GEO occurred 10 minutes after the new processors were in place. Any other ideas that would explain it?

-dulci

[gnwiii]

There could be a transient network glitch,  a problem with the update or a change in the python program used with modis_geo.py.   Please provide details of your platform (we can assume macOS, but version may help someone recognize your problem).   Since we know the Apple supplied python2 uses an obsolete openssl library, we can assume you used a different python to run the  command-line install, so it would be good to mention which python you used.  To check that the update succeeded, run:
$ cd $OCSSWROOT/run/scripts
$ git log (where the $ represents the command prompt in Terminal.
Normally, the first line of geogen modis is:
$ head -1 modis_GEO.py
#! /usr/bin/env python
You can check which python is specified in your PATH by running:
$ which python
You can also specify which python is used:
$ <full path to your python2> $OCSSWROOT/run/scripts/modis_GEO.py -v --log /Users/dulci/Documents/MODIS/LakeErie2011/A2011246185500.L1A_LAC
It could also help to provide the output from:
$ /usr/bin/env python --version as this is probably the version being used by modis_GEO.py

[davouris]

Would the update have failed if the links to the processors in the GUI SeaDAS used to be light grey & didn't work, but now they link to the window that lets me run the process?

I am on a macOS El capitan 10.11.6. I installed python 2.7.13 using the command line installation from anaconda. I do not code in python. I have a fairly basic understanding of bash/ unix commands. I use SeaDAS for first and second level image pre-processing before I import it into IDL/ENVI for my research analysis.

Riesling:~ dulci$ python
Python 2.7.13 |Anaconda 4.3.0 (x86_64)| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org

I moved the OCSSW folder that was installed when I ran the install_ocssw.py script into my SeaDAS folder in my applications folder because the processors didn't work on the GUI platform before I did that. The OCSSWROOT/run/scripts apparently doesn't exist on my machine.

Riesling:~ dulci$ cd $OCSSWROOT/run/scripts
-bash: cd: /run/scripts: No such file or directory

however, I can navigate to that folder using the file finder, and look at all the scripts, etc.

The SeaDAS folder is where the installer put it: MacintoshHD/Applications/seadas-7.3.1. I don't know how to navigate to MacintoshHD from my home directory of MacintoshHD/users/dulci, or how to specify that path as the location where the ocssw should be installed.  Is there another way to get to that folder besides cd .. from my home to go up a few levels?

Do I need to run the install_ocssw.py script again with more specific information? I let the script use the default dir.

The python path specified by which python is:

Riesling:~ dulci$ which python
/Users/dulci/anaconda2/bin/python

And the output from the /usr/bin/env python is:

Riesling:~ dulci$ /usr/bin/env python --version
Python 2.7.13 :: Anaconda 4.3.0 (x86_64)

Is the problem that SeaDAS and the OCSSW processors are two levels up from my python path?

Or could this be the same problem encountered by people using the direct broadcast, which is that the lines of code specifying the entrained flag are commented out?

This was definitely a birdshot message. I appreciate the help. I have enough knowledge to be dangerous, but not enough to really understand what is going on.

-dulci

[OB.DAAC - SeanBailey]

The version of python (via anaconda) is fine.

$OCSSWROOT is an environment variable that should point to the ocssw directory in your SeaDAS install path, it is not automatically set in your shell, so you would have to define it (e.g. export OCSSWROOT=/Applications/seadas-7.3.1/ocssw).  You would then want to source the $OCSSWROOT/OCSSW_bash.env file (e.g. source $OCSSWROOT/OCSSW_bash.env).

How did you call install_ocssw.py when you finally got it to work?
If you just need the MODIS-Aqua processing capability, you could try:

$ install_ocssw.py --install-dir=/Applications/seadas-7.3.1/ocssw --git-branch=v7.3 --aqua (the leading $ indicates the shell prompt, no need to type it)

Oh, and you may want to update the GUI to 7.3.2...or wait a little while and jump to 7.4 which we anticipate releasing soon.

Sean

[gnwiii]

The fact that the processors are no longer grayed out indicates that the GUI is able to find them.  By default, the GUI puts OCSSWROOT in the top-level SeaDAS folder, while the manual installer puts it in your "home" directory.  Yours seems to have ended up in the top-level SeaDAS folder, so you would use OCSSWROOT=/Applications/seadas-7.3.1/ocssw, after verifying that there isn't a second ocssw folder in your home directory. 

The error:
Riesling:~ dulci$ cd $OCSSWROOT/run/scripts
-bash: cd: /run/scripts: No such file or directory
indicates that the OCSSWROOT variable is not set in the bash shell.  At the bottom of the manual installation page is a section on "Command line configuration".  This will add the OCSSWROOT variable to the bash startup scripts and set other variables needed to run processors on the command line.  Open a new terminal after editing your .bashrc file and check that cd $OCSSWROOT/run/scripts works. 

The important non-default installer options are the install directory, the git branch, which should be "-b v7.3", and the sensors you use.

SeaDAS is a very complex system and the ESA and NASA people have done a great job of hiding much of the complexity, but unfortunately the https change
relies on tools provided by the OS vendor and in this case those tools (e.g., Apple Python 2.7, Ubuntu 14.04 git) are makign thigs too easy for those bad people exploiting insecure connections (e.g., http), leaving users to fend for themselves.  Imagine that bad people found a simple way to make keys to your car and the car maker isn't helping you install better locks.

[davouris]

Hi Sean,

When I got the install_ocssw.py to run I just typed $ python install_ocssw.py with the flags for aqua, terra, meris & hico. I will try again with the install dir flag as well.
I think the problem before was when I copy and pasted the install_ocssw.py script from the webpage to a text editor an extra line or character or something got in there. I deleted the first version and tried again and got a clean script.

My SeaDAS GUI says 7.3.2 on the load screen and at the top of the window, but my folder says 7.3.1. Does that matter? I try to stay on top of those because I use it frequently.

I just did the export OCSSWROOT command and am starting the install_ocssw.py again, with the install-dir flag and git-branch flag set as you suggested. I'll see how it works this time.

Thank you for all the help!

-dulci

[davouris]

Good morning!

I reinstalled the OCSSW processors in my seadas directory this morning, and as far as I can tell that all worked well. No error messages in the terminal. I used the anaconda python command line.

The links in my SeaDAS 7.3.2 GUI are lit up and I get the modis_GEO window. My L1A file is loaded in SeaDAS, so that is the input file. I did not specify any of the parameters listed in that window. This is the error I get when I run the processor:

execution exception: java.io.IOException: modis_GEO.py failed with exit code 1.
Check log for more details.
Error! could not establish a network connection. Check your network connection.
If you do not find a problem, please try again later.
Determining required attitude and ephemeris files...

Input file: /Users/dulci/Documents/MODIS/LakeErie2011/A2011246185500.L1A_LAC
Start time: 2011246185500
End time: 2011246185959

Cannot create geolocation from /Users/dulci/Documents/MODIS/LakeErie2011/A2011246185500.L1A_LAC; exiting.

The python I am using is the anaconda python, and that version also shows up when I check the /usr/bin/env python.
I got the log when I went did this:

$ cd $OCSSWROOT/run/scripts
$ git log

So I'm pretty sure everything is in the right place. Should I uncomment the lines in the modis_GEO code about the entrainment?

What am I doing wrong now?

thank you!

-dulci

[OB.DAAC - SeanBailey]

> Error! could not establish a network connection. Check your network connection

This message is where we should be focusing.  Lets start simple.  From a command-line try the following:

$ cd $OCSSWROOT/run/scripts
$ git status

That *should* show:
On branch v7.3
Your branch is up-to-date with 'origin/v7.3'.
nothing to commit, working tree clean

If that is so, try running the modis_GEO.py script from that terminal:
$ cd  /Users/dulci/Documents/MODIS/LakeErie2011/
$ modis_GEO.py --verbose A2011246185500.L1A_LAC

Sean