l2bin error

[OB SeaDAS - dshea]

That is kind of strange.  I looked through your script for environment variables that might screw things up, but it looks fine to me.

don

[obdaac_forum_user]

I can't see what is wrong with the code either.  I actually did not write it.  Another forum member blesht did.  He was kind enough to share it with me!

What is worse is I am only using the workflow I am using (L2 OC files > l2bin > L3b files > l3mapgen) because I can't figure out how to use gpt.command to reproject many files and save to files with all bands that I can use in R to calculate chlor_a with a Great Lakes-specific algorithm (R workflow uses raster package).  Would prefer, I think, to use L2 data anyway.

Maybe I need a new workflow.  Too bad as the existing R code works nicely. 

Thanks,
Dave

[gnwiii]

You can certainly encounter problems with environment settings when mixing shell programs.   There are some security issues with bash: CVE-2016-0634, CVE-2016-7543, CVE-2016-9401, CVE-2017-5932 were just mentioned in Ubuntu Security Notice USN-3294-1, but Apple is slow and doen't always tell us which CVE's are being patched so in addition to the new feaures in bash 4, security concerns may be reasons to use a recent version.

Have you seen Upgrade to bash 4 in macOS?   This illustrates a few ways to manage the shell version used in a particular script.    On recent versions of MacOS, Apple System Integrity Protection should prevent replacing /bin/bash with a symbolic link to /usr/local/bin/bash.   It is, however, possible to disable SIP.   The first line of your script reads "#!/bin/bash", so if you are using bash 4 in a terminal the script could be getting different settings.  From the examples in the "Upgrade to bash 4 " link you should be able to devise some scirpts to display key varaibles in
different shell environments.

If l2bin works from a shell prompt that points to some difference in the environment between the script and the prompt.  You can start from a very simple script and add back the extra parameter passing and checking logic  to see if you can figure out where things go wrong.

[obdaac_forum_user]

Thanks gnwii!  That link and is what I used to install bash 4 as far as I have.  It is installed but I've yet been able to make it the default shell.  Perhaps I can figure out how to make the script use bash 4.  I have been getting a message that says "/usr/local/bin/bash does not exist" in this process and I have no clue where homebrew installed bash 4 so that I can direct my script to it.

Dave

[gnwiii]

Better to stick with the default Apple configuration until things are working since that is what most other people will have.  I haven't used homebrew in years, but I recall that the default was to install in a "private" area and create symbolic links under /usr/local.   There seems to something about your system that is "non-standard", either some missing or corrupt file in the ocssw directories or something (usually environment variable) causing the software to look for files in the wronge place.   That can easily happen if you have previousy installed seadas and forget to remove old environment settings when you install a newer version.  It has happened to many of us in my lab and can be tedious to resolve, even with the advantages of several pairs of eyes and side-by-side comparisons with a working system.

[obdaac_forum_user]

Thanks again!  Feeling worried now, more than anything else I suppose.

Not sure how I will get to the point of "things working".  And I wish I knew much earlier that issues with installation can have such far reaching impacts.  I would have taken greater care.

One thing that is nonstandard about my system is that it is a Mac.  That might be standard where you work, but is not where I work (USGS) and the implementation of security and other policies is not, in my opinion, well thought out even for Windows systems.  And the support for Macs, so far, has not been good.

This feature of my system, is, in all likelihood, part of the puzzle.

Do you mean that there is something causing ocssw to look for files in the wrong place?

In the meantime, I've modified my R code that uses L2 OC data and the raster and plotKML packages to estimate chlor_a so that it now also maps the estimated chlor_a.  While it is likely a different mapping procedure than is employed by OBPG products, it is still based on the principle of resampling and I can provide the code to anyone who wants to use it.  As long as the packages I used persist anyway.

Cheers,
Dave

[gnwiii]

For every forum post like yours there are many other users who just give up and rely on the standard NASA products and most problems encountered by forum users show up sooner of later in our lab.

I suggest you set up a new "ocssw" account andt install using only the command-line (the default location will be ~/ocssw).  In the long run, having such a bare-bones account is often helpful for troubleshooting other issues. If you still having network problems, follow the offline installation instructions (download the git bundles and use the --local=<bundle_directory> option as well as the appropriate branch and sensor).  You  will need to use Anaconda python 2.7to run the installer and add lines to set the OCSSW variables to the "/Users/ocssw/.bashrc".  Logout and back in to get the settings in a terminal and run update_luts.py to make sure everything is current.  It is very likely that will get l2bin going, itf not the next step would be a basic install of macOS (e.g., on an external disk).

Once you have a configuration that runs l2bin without the segmentation faults, you can proceed with your processing and track down the difference at leisure.

[obdaac_forum_user]

I really appreciate you providing all the suggestions.  That sounds like a good plan to try.  Hopefully none of it will require admin authority or if required, it could be done in one sitting.  If it requires assistance from anyone with admin AND more experience than me then it could take a while. 

I do have R processing the stuff ok but slowly.  Something close to 30 seconds per scene with parallelization using foreach().

[gnwiii]

Adding an ocssw user will require an admin, but only for a few minutes unless your site's policies require an extensive paperwork exercise for such accounts.