command line OCSSW advice

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 258
Joined: Thu Mar 05, 2009 10:25 am America/New_York
Answers: 0
Been thanked: 2 times

command line OCSSW advice

by OB SeaDAS - dshea » Mon Mar 16, 2020 8:34 pm America/New_York

Not sure how the TileSize attribute got in your L2 file, but the flags line up easily:

l2_flags:flag_meanings = "ATMFAIL LAND PRODWARN HIGLINT HILT HISATZEN COASTZ SPARE8 STRAYLIGHT CLDICE COCCOLITH TURBIDW HISOLZEN SPARE14 LOWLW CHLFAIL NAVWARN ABSAER SPARE19 MAXAERITER MODGLINT CHLWARN ATMWARN SPARE24 SEAICE NAVFAIL FILTER SPARE28 BOWTIEDEL HIPOL PRODFAIL SPARE32" ;

l2_flags:flag_masks = 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 16777216, 33554432, 67108864, 134217728, 268435456, 536870912, 1073741824, -2147483648

Therefore:
ATMFAIL = bit 1 = 1
LAND = bit 2 = 2
PRODWARN = bit 3 = 4
HIGLINT = bit 4 = 8
HILT = bit 5 = 16
...

don

Tags:

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

command line OCSSW advice

by gnwiii » Mon Mar 16, 2020 8:49 pm America/New_York

DOCS/ALGORITHM DESCRIPTIONS  has slightly more detail on level-2 flags, including a tool to compute masks.

I've never noticed "TileSize" in the metadata, and don't see it in the metadata examples.   TIFF files can use tiles. I didn't think the USGS GeoTIFF's used tiles, but that metadata entry might have been inherited from the input level-1 file.

avmehta
Posts: 184
Joined: Mon Feb 03, 2020 10:27 am America/New_York
Answers: 0
Been thanked: 1 time

command line OCSSW advice

by avmehta » Tue Mar 17, 2020 11:45 am America/New_York

Thanks a lot for the link.
Amita

avmehta
Posts: 184
Joined: Mon Feb 03, 2020 10:27 am America/New_York
Answers: 0
Been thanked: 1 time

command line OCSSW advice

by avmehta » Tue Mar 17, 2020 11:52 am America/New_York

Thanks - yes, I know how they lineup. My question is about definitions of some the flags -- most are self-explanatory but I am somewhat new to this so do not know some other flags, e.g. COCCOLITH. Also, I see  'FILTER', 'BOWTIEDEL' (is this to do with swath?), 'HIPOL' with mask values. Not sure what they mean and how the mask values are decided.
Thanks for your help.
Amita

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

command line OCSSW advice

by gnwiii » Wed Mar 18, 2020 1:13 pm America/New_York

I'll use COCCOLITH as an example.   You can read the MODIS Algorithm Theoretical Basis Document 23.    To see how the flag is set in l2gen, install the source code (using the install_ocssw.py script or the SeaDAS GUI) and have a look at for the following section:
char isCoccolith(l2str *l2rec, int32_t ip) {
    static float firstCall = 1;
    static float c1, c2, c3, c4, c5, c6, c7, c8;
    int32_t nbands = l2rec->l1rec->l1file->nbands;

    float nLw443 = l2rec->nLw[ip * nbands + ib443];
    float nLw510 = l2rec->nLw[ip * nbands + ib510];
    float nLw555 = l2rec->nLw[ip * nbands + ib555];
    float La670 = l2rec->La [ip * nbands + ib670];

    if (firstCall) {
        firstCall = 0;
        c1 = input->coccolith[0];
        c2 = input->coccolith[1];
        c3 = input->coccolith[2];
        c4 = input->coccolith[3];
        c5 = input->coccolith[4];
        c6 = input->coccolith[5];
        c7 = input->coccolith[6];
        c8 = input->coccolith[7];
    }

    if (nLw443 >= c1 &&
            nLw510 >= 0.0 &&
            nLw555 >= c2 &&
            La670 <= 1.1 &&
            Between(c3, nLw443 / nLw555, c4) &&
            Between(c5, nLw510 / nLw555, c6) &&
            Between(c7, nLw443 / nLw510, c8))

        return (1);
    else
        return (0);
}

Definitions of the constants (c1,...,c8) are an "exercise left to the reader", as is finding where the COCCOLITH flag is set.   In general, details of how a flag is set can be found by reading ATBD's and processing histories in combination with the source code.   If you are comfortable compiling the processing programs from source, it can be helpful to add some print statements so you can see the values of the variables you are looking at. 

If you get stuck, there should  be others on this forum who can point you in the right direction.

avmehta
Posts: 184
Joined: Mon Feb 03, 2020 10:27 am America/New_York
Answers: 0
Been thanked: 1 time

command line OCSSW advice

by avmehta » Wed Apr 01, 2020 6:55 pm America/New_York

Hello,
   I am  new to using SeaDAS/OCSSW on Linux. I am trying to install OCSSW on my linux sever for msis2a and msis2b.  I use the following:
./install_ocssw.py --install-dir=$home/ocssw --git-branch=v1.7.1 -c  --msis2a --msis2b
I assumed that the git-branch would ve v7.5 or 7.5.3 (SeaDAS version number). But that does not work so I used the git version number!
I get the following error:

2020-04-01 18:33:24 (62.9 MB/s) - “common.bundle” saved [1270826692/1270826692]

Initialized empty Git repository in /home/avmehta/ocssw/run/data/common/.git/
error: unknown switch `B'
usage: git checkout [options] <branch>
   or: git checkout [options] [<branch>] -- <file>...

    -q, --quiet           be quiet
    -b <new branch>       branch
    -l                    log for new branch
    -t, --track           track
    -2, --ours            stage
    -3, --theirs          stage
    -f, --force           force
    -m, --merge           merge
    --conflict <style>    conflict style (merge or diff3)
    -p, --patch           select hunks interactively

Error - Could not run "cd /home/avmehta/ocssw/run/data/common; git checkout -t -B v1.7.1 remotes/origin/v1.7.1 -q > /dev/null"

Could you please help?
Thanks.
Amita

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

command line OCSSW advice

by gnwiii » Wed Apr 01, 2020 7:17 pm America/New_York

You should use --git-branch=v7.5.   Since that didn't work, check that you have the most recent install_ocssw.py and that your git version meets the requirements.  As I recall, the "-B" switch is relatively new, so your git version may need updating.  On my system the man page for git-checkout has:

GIT-CHECKOUT(1)                   Git Manual                   GIT-CHECKOUT(1)

NAME
       git-checkout - Switch branches or restore working tree files

SYNOPSIS
       git checkout [-q] [-f] [-m] [<branch>]
       git checkout [-q] [-f] [-m] --detach [<branch>]
       git checkout [-q] [-f] [-m] [--detach] <commit>
       git checkout [-q] [-f] [-m] [[-b|-B|--orphan] <new_branch>] [<start_point>]
       git checkout [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>] [--] <pathspec>...
       git checkout [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>] --pathspec-from-file=<file> [--pathspec-file-nul]
       git checkout (-p|--patch) [<tree-ish>] [--] [<pathspec>...]


It might be useful to mention your linux distro and version in case someone else has encountered the same problem.

avmehta
Posts: 184
Joined: Mon Feb 03, 2020 10:27 am America/New_York
Answers: 0
Been thanked: 1 time

command line OCSSW advice

by avmehta » Thu Apr 02, 2020 11:10 am America/New_York

Thank you for the reply (and also for  my  previous post). I will try updating git first.

My OS information is:

Distributor ID:  CentOS
Description:  CentOS release 6.10 (Final)

Thanks.
Amita

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

command line OCSSW advice

by gnwiii » Thu Apr 02, 2020 12:43 pm America/New_York

You will probably need to update other packages, which can become tedious.   The OCSSW binaries were being built on CentOS 6, but if you need to download ancillary data there were recent changes to the Python scripts using the Python "requests" library.

avmehta
Posts: 184
Joined: Mon Feb 03, 2020 10:27 am America/New_York
Answers: 0
Been thanked: 1 time

command line OCSSW advice

by avmehta » Mon Apr 06, 2020 4:12 pm America/New_York

Thank you.
With the help of our branch's system administrator, OCSSW was installed on my linux machine!
Now when I run it for an OLI L1 image I get the following errors:

OCDATAROOT environment variable is not defined.
-E- /home/seadas/ocssw/src/l2gen/msl12_input.c: Error reading program options.
-E- l2gen: Error parsing input parameters.

I read in the Forum posts about this and  defined OCSSWROOT and also OCDATAROOT (according OCSSW_bash.env) but I still get the same error.
My confusion is about the second line : -E- /home/seadas/ocssw/src/l2gen/msl12_input.c: Error reading program options.
I have no such directory as /home/seadas/ocssw/  because I have installed seadas_7.5.3 on my /home/avmehta director and also ocssw on /home/avmehta.
I am using tcsh BTW.
Any advice?
Thanks.
Amita

Post Reply