Getting started with the OCSSW Command line

[emaure]

Hi folks,

This may be a simple question but, I am trying to get started with ocssw command line data processing.
For now, final goal is to be able to generate composites (temporal) using l3bin.
As I never used these command line tools before, I started with examples provided in this post https://seadas.gsfc.nasa.gov/help/OcsswCookbook/ocsswCookbookMappedSst.html.
However, right from beginning I am getting total_filled_bins: 0.

Is there anything I am doing wrong?
I installed the OCSSW using the install_ocssw.py

My PC specs are shown below

>CentOS Linux 7
>Memory 15.3 GiB
>Processor Intel Core i7-8550U CPU @ 1.8GHZ x 8
>OS Type 64-Bit

I call the l2bin to generate a binned file from L2 as

> ~/ocssw/bin/l2bin ifile=/home/eligio/Documents/Level_2/VIIRS-SNPP/2018/SNPP_VIIRS.20180101T172401.L2.SST.nc ofile=SNPP_VIIRS.20180101T17240.L3b_SST_StdQual_1km.nc par=l2bin_SST_StdQual.par

l2bin_SST_StdQual.par

> prodtype=regional
> resolution=1
> qual_prod=qual_sst
> qual_max=2
> l3bprod=sst
> flaguse=LAND,HISOLZEN

The response from l2bin is as shown below
2bin 5.0.0 (Aug 16 2019 12:47:25)
Loading default parameters from /home/eligio/ocssw/share/viirs/l2bin_defaults.par
Loading command line parameters

INTERP parameter disabled.
Single HDF input
Averaging: standard
Resolution: 1
Max Qual Allowed: 2
prod_avg:
flagusemask: 4098
required: 0
NetCDF4 input file
/home/eligio/Documents/Level_2/VIIRS-SNPP/2018/SNPP_VIIRS.20180101T172401.L2.SST.nc   brk:    0   3232   1  62641
total number of bins: 380187130
1970001 2038018 270
krow:     0 out of  17280  (-90.00 to -87.19) Fri Feb  7 16:18:17 2020
krow:   270 out of  17280  (-87.19 to -84.38) Fri Feb  7 16:18:17 2020
krow:   540 out of  17280  (-84.38 to -81.56) Fri Feb  7 16:18:17 2020
krow:   810 out of  17280  (-81.56 to -78.75) Fri Feb  7 16:18:17 2020
krow:  1080 out of  17280  (-78.75 to -75.94) Fri Feb  7 16:18:17 2020
krow:  1350 out of  17280  (-75.94 to -73.12) Fri Feb  7 16:18:17 2020
krow:  1620 out of  17280  (-73.12 to -70.31) Fri Feb  7 16:18:17 2020
krow:  1890 out of  17280  (-70.31 to -67.50) Fri Feb  7 16:18:17 2020
krow:  2160 out of  17280  (-67.50 to -64.69) Fri Feb  7 16:18:17 2020
krow:  2430 out of  17280  (-64.69 to -61.88) Fri Feb  7 16:18:17 2020
krow:  2700 out of  17280  (-61.88 to -59.06) Fri Feb  7 16:18:17 2020
krow:  2970 out of  17280  (-59.06 to -56.25) Fri Feb  7 16:18:17 2020
krow:  3240 out of  17280  (-56.25 to -53.44) Fri Feb  7 16:18:17 2020
krow:  3510 out of  17280  (-53.44 to -50.62) Fri Feb  7 16:18:17 2020
krow:  3780 out of  17280  (-50.62 to -47.81) Fri Feb  7 16:18:17 2020
krow:  4050 out of  17280  (-47.81 to -45.00) Fri Feb  7 16:18:17 2020
krow:  4320 out of  17280  (-45.00 to -42.19) Fri Feb  7 16:18:17 2020
krow:  4590 out of  17280  (-42.19 to -39.38) Fri Feb  7 16:18:17 2020
krow:  4860 out of  17280  (-39.38 to -36.56) Fri Feb  7 16:18:17 2020
krow:  5130 out of  17280  (-36.56 to -33.75) Fri Feb  7 16:18:17 2020
krow:  5400 out of  17280  (-33.75 to -30.94) Fri Feb  7 16:18:17 2020
krow:  5670 out of  17280  (-30.94 to -28.12) Fri Feb  7 16:18:17 2020
krow:  5940 out of  17280  (-28.12 to -25.31) Fri Feb  7 16:18:17 2020
krow:  6210 out of  17280  (-25.31 to -22.50) Fri Feb  7 16:18:17 2020
krow:  6480 out of  17280  (-22.50 to -19.69) Fri Feb  7 16:18:17 2020
krow:  6750 out of  17280  (-19.69 to -16.88) Fri Feb  7 16:18:17 2020
krow:  7020 out of  17280  (-16.88 to -14.06) Fri Feb  7 16:18:17 2020
krow:  7290 out of  17280  (-14.06 to -11.25) Fri Feb  7 16:18:17 2020
krow:  7560 out of  17280  (-11.25 to  -8.44) Fri Feb  7 16:18:17 2020
krow:  7830 out of  17280  ( -8.44 to  -5.62) Fri Feb  7 16:18:17 2020
krow:  8100 out of  17280  ( -5.62 to  -2.81) Fri Feb  7 16:18:17 2020
krow:  8370 out of  17280  ( -2.81 to   0.00) Fri Feb  7 16:18:17 2020
krow:  8640 out of  17280  (  0.00 to   2.81) Fri Feb  7 16:18:17 2020
krow:  8910 out of  17280  (  2.81 to   5.62) Fri Feb  7 16:18:17 2020
krow:  9180 out of  17280  (  5.62 to   8.44) Fri Feb  7 16:18:17 2020
krow:  9450 out of  17280  (  8.44 to  11.25) Fri Feb  7 16:18:17 2020
krow:  9720 out of  17280  ( 11.25 to  14.06) Fri Feb  7 16:18:17 2020
krow:  9990 out of  17280  ( 14.06 to  16.88) Fri Feb  7 16:18:17 2020
krow: 10260 out of  17280  ( 16.88 to  19.69) Fri Feb  7 16:18:17 2020
krow: 10530 out of  17280  ( 19.69 to  22.50) Fri Feb  7 16:18:17 2020
krow: 10800 out of  17280  ( 22.50 to  25.31) Fri Feb  7 16:18:17 2020
krow: 11070 out of  17280  ( 25.31 to  28.12) Fri Feb  7 16:18:18 2020
krow: 11340 out of  17280  ( 28.12 to  30.94) Fri Feb  7 16:18:18 2020
krow: 11610 out of  17280  ( 30.94 to  33.75) Fri Feb  7 16:18:19 2020
krow: 11880 out of  17280  ( 33.75 to  36.56) Fri Feb  7 16:18:21 2020
krow: 12150 out of  17280  ( 36.56 to  39.38) Fri Feb  7 16:18:22 2020
krow: 12420 out of  17280  ( 39.38 to  42.19) Fri Feb  7 16:18:24 2020
krow: 12690 out of  17280  ( 42.19 to  45.00) Fri Feb  7 16:18:38 2020
krow: 12960 out of  17280  ( 45.00 to  47.81) Fri Feb  7 16:18:40 2020
krow: 13230 out of  17280  ( 47.81 to  50.62) Fri Feb  7 16:18:42 2020
krow: 13500 out of  17280  ( 50.62 to  53.44) Fri Feb  7 16:18:44 2020
krow: 13770 out of  17280  ( 53.44 to  56.25) Fri Feb  7 16:18:44 2020
krow: 14040 out of  17280  ( 56.25 to  59.06) Fri Feb  7 16:18:44 2020
krow: 14310 out of  17280  ( 59.06 to  61.88) Fri Feb  7 16:18:44 2020
krow: 14580 out of  17280  ( 61.88 to  64.69) Fri Feb  7 16:18:44 2020
krow: 14850 out of  17280  ( 64.69 to  67.50) Fri Feb  7 16:18:44 2020
krow: 15120 out of  17280  ( 67.50 to  70.31) Fri Feb  7 16:18:44 2020
krow: 15390 out of  17280  ( 70.31 to  73.12) Fri Feb  7 16:18:44 2020
krow: 15660 out of  17280  ( 73.12 to  75.94) Fri Feb  7 16:18:44 2020
krow: 15930 out of  17280  ( 75.94 to  78.75) Fri Feb  7 16:18:44 2020
krow: 16200 out of  17280  ( 78.75 to  81.56) Fri Feb  7 16:18:44 2020
krow: 16470 out of  17280  ( 81.56 to  84.38) Fri Feb  7 16:18:44 2020
krow: 16740 out of  17280  ( 84.38 to  87.19) Fri Feb  7 16:18:44 2020
krow: 17010 out of  17280  ( 87.19 to  90.00) Fri Feb  7 16:18:44 2020
krow:17280 Fri Feb  7 16:18:44 2020
total_filled_bins: 0
rm -f SNPP_VIIRS.20180101T17240.L3b_SST_StdQual_1km.nc*

Thank you
eligio

[OB.DAAC - SeanBailey]

eligio,

The file is a nighttime granule:
$ ncdump -h SNPP_VIIRS.20180101T172401.L2.SST.nc |grep day_night_flag
                :day_night_flag = "Night" ;

The flaguse option you set excludes nighttime data.  You want the inverse of that:
flaguse=LAND,~HISOLZEN
or just reduce your parameter options to

resolution=1
suite=NSST

Sean

[gnwiii]

Your level-2 file has very few pixels with quality 1 or 2.  On my system (Fedora 31), l2bin complained that "row_group" was not defined, but after adding "rowgroup=50"  to the par file it also gave 0 bins filled.   Normally, when there are bins with valid data the output will have lines like:

# bins in row group : 3131973
# filled bins       :    1342


Have you tried reproducing the examples in the tutorial?   Ideally you would use the same input file, but there is a new reprocessing:

Original File: https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/V2015045190600.L2_SNPP_SST.nc
New File: https://oceandata.sci.gsfc.nasa.gov/ob/getfile/SNPP_VIIRS.20150214T190601.L2.SST.nc

If you can reproduce the examples this should give you confidence that your OCSSW software is working properly.

[emaure]

Thank you Sean and George for your reply.

Thanks for providing the new file name. It helps keeping consistent with the example as I initially planned to do so.

After your advises, I run l2bin for my nighttime SST reversing the flag and for the daytime SST with the original setting.
Both options give me an empty "*.nc" file and complain that "-E- OCVARROOT environment variable is not defined"
Maybe I missed something, but I do not find any explanation about setting this variable.

Question: How to set this OCVARROOT? Or is this issue related to any problem in my installation?

Below is the full report from l2bin
l2bin 5.0.0 (Aug 16 2019 12:47:25)
Loading default parameters from /home/eligio/ocssw/share/viirs/l2bin_defaults.par
Loading command line parameters

INTERP parameter disabled.
Single HDF input
Averaging: standard
Resolution: 1
Max Qual Allowed: 2
prod_avg:
flagusemask: 4098
required: 0
NetCDF4 input file
/home/eligio/Downloads/SNPP_VIIRS.20150214T190601.L2.SST.nc   brk:    0   3232  45  68761
total number of bins: 380187130
1970001 2038018 270
krow:     0 out of  17280  (-90.00 to -87.19) Wed Feb 19 12:02:58 2020
krow:   270 out of  17280  (-87.19 to -84.38) Wed Feb 19 12:02:58 2020
krow:   540 out of  17280  (-84.38 to -81.56) Wed Feb 19 12:02:58 2020
krow:   810 out of  17280  (-81.56 to -78.75) Wed Feb 19 12:02:58 2020
krow:  1080 out of  17280  (-78.75 to -75.94) Wed Feb 19 12:02:58 2020
krow:  1350 out of  17280  (-75.94 to -73.12) Wed Feb 19 12:02:58 2020
krow:  1620 out of  17280  (-73.12 to -70.31) Wed Feb 19 12:02:58 2020
krow:  1890 out of  17280  (-70.31 to -67.50) Wed Feb 19 12:02:58 2020
krow:  2160 out of  17280  (-67.50 to -64.69) Wed Feb 19 12:02:58 2020
krow:  2430 out of  17280  (-64.69 to -61.88) Wed Feb 19 12:02:58 2020
krow:  2700 out of  17280  (-61.88 to -59.06) Wed Feb 19 12:02:58 2020
krow:  2970 out of  17280  (-59.06 to -56.25) Wed Feb 19 12:02:58 2020
krow:  3240 out of  17280  (-56.25 to -53.44) Wed Feb 19 12:02:58 2020
krow:  3510 out of  17280  (-53.44 to -50.62) Wed Feb 19 12:02:58 2020
krow:  3780 out of  17280  (-50.62 to -47.81) Wed Feb 19 12:02:58 2020
krow:  4050 out of  17280  (-47.81 to -45.00) Wed Feb 19 12:02:58 2020
krow:  4320 out of  17280  (-45.00 to -42.19) Wed Feb 19 12:02:58 2020
krow:  4590 out of  17280  (-42.19 to -39.38) Wed Feb 19 12:02:58 2020
krow:  4860 out of  17280  (-39.38 to -36.56) Wed Feb 19 12:02:58 2020
krow:  5130 out of  17280  (-36.56 to -33.75) Wed Feb 19 12:02:58 2020
krow:  5400 out of  17280  (-33.75 to -30.94) Wed Feb 19 12:02:58 2020
krow:  5670 out of  17280  (-30.94 to -28.12) Wed Feb 19 12:02:58 2020
krow:  5940 out of  17280  (-28.12 to -25.31) Wed Feb 19 12:02:58 2020
krow:  6210 out of  17280  (-25.31 to -22.50) Wed Feb 19 12:02:58 2020
krow:  6480 out of  17280  (-22.50 to -19.69) Wed Feb 19 12:02:58 2020
krow:  6750 out of  17280  (-19.69 to -16.88) Wed Feb 19 12:02:58 2020
krow:  7020 out of  17280  (-16.88 to -14.06) Wed Feb 19 12:02:58 2020
krow:  7290 out of  17280  (-14.06 to -11.25) Wed Feb 19 12:02:58 2020
krow:  7560 out of  17280  (-11.25 to  -8.44) Wed Feb 19 12:02:58 2020
krow:  7830 out of  17280  ( -8.44 to  -5.62) Wed Feb 19 12:02:58 2020
krow:  8100 out of  17280  ( -5.62 to  -2.81) Wed Feb 19 12:02:58 2020
krow:  8370 out of  17280  ( -2.81 to   0.00) Wed Feb 19 12:02:58 2020
krow:  8640 out of  17280  (  0.00 to   2.81) Wed Feb 19 12:02:58 2020
krow:  8910 out of  17280  (  2.81 to   5.62) Wed Feb 19 12:02:58 2020
krow:  9180 out of  17280  (  5.62 to   8.44) Wed Feb 19 12:02:58 2020
krow:  9450 out of  17280  (  8.44 to  11.25) Wed Feb 19 12:02:58 2020
krow:  9720 out of  17280  ( 11.25 to  14.06) Wed Feb 19 12:02:58 2020
krow:  9990 out of  17280  ( 14.06 to  16.88) Wed Feb 19 12:02:58 2020
krow: 10260 out of  17280  ( 16.88 to  19.69) Wed Feb 19 12:02:58 2020
krow: 10530 out of  17280  ( 19.69 to  22.50) Wed Feb 19 12:03:01 2020
-E- OCVARROOT environment variable is not defined.

Thank you for your help.
Eligio

[OB SeaDAS - dshea]

looks like your OCSSW is installed at "~/ocssw".  You need to do these steps to setup the environment variables for the processing programs to work properly.

export OCSSWROOT=~/ocssw
source $OCSSWROOT/OCSSW_bash.env

Then you can run the programs without specifying the whole path.

don

[emaure]

Many thanks Don,

I think I now have the OCSSW up and running.
Just tested the SST file and worked fine as shown in the report below.
Will now run the other exercises in the cookbook to familiarize with the command line.

l2bin 5.0.0 (Aug 16 2019 12:47:25)
Loading default parameters from /home/eligio/ocssw/share/viirs/l2bin_defaults.par
Loading command line parameters

INTERP parameter disabled.
Single HDF input
Averaging: standard
Resolution: 1
Max Qual Allowed: 2
prod_avg:
flagusemask: 4098
required: 0
NetCDF4 input file
/home/eligio/Downloads/SNPP_VIIRS.20150214T190601.L2.SST.nc   brk:    0   3232  45  68761
total number of bins: 380187130
1970001 2038018 270
krow:     0 out of  17280  (-90.00 to -87.19) Thu Feb 20 10:04:25 2020
krow:   270 out of  17280  (-87.19 to -84.38) Thu Feb 20 10:04:25 2020
krow:   540 out of  17280  (-84.38 to -81.56) Thu Feb 20 10:04:25 2020
krow:   810 out of  17280  (-81.56 to -78.75) Thu Feb 20 10:04:25 2020
krow:  1080 out of  17280  (-78.75 to -75.94) Thu Feb 20 10:04:25 2020
krow:  1350 out of  17280  (-75.94 to -73.12) Thu Feb 20 10:04:25 2020
krow:  1620 out of  17280  (-73.12 to -70.31) Thu Feb 20 10:04:25 2020
krow:  1890 out of  17280  (-70.31 to -67.50) Thu Feb 20 10:04:25 2020
krow:  2160 out of  17280  (-67.50 to -64.69) Thu Feb 20 10:04:25 2020
krow:  2430 out of  17280  (-64.69 to -61.88) Thu Feb 20 10:04:25 2020
krow:  2700 out of  17280  (-61.88 to -59.06) Thu Feb 20 10:04:25 2020
krow:  2970 out of  17280  (-59.06 to -56.25) Thu Feb 20 10:04:25 2020
krow:  3240 out of  17280  (-56.25 to -53.44) Thu Feb 20 10:04:25 2020
krow:  3510 out of  17280  (-53.44 to -50.62) Thu Feb 20 10:04:25 2020
krow:  3780 out of  17280  (-50.62 to -47.81) Thu Feb 20 10:04:25 2020
krow:  4050 out of  17280  (-47.81 to -45.00) Thu Feb 20 10:04:25 2020
krow:  4320 out of  17280  (-45.00 to -42.19) Thu Feb 20 10:04:25 2020
krow:  4590 out of  17280  (-42.19 to -39.38) Thu Feb 20 10:04:25 2020
krow:  4860 out of  17280  (-39.38 to -36.56) Thu Feb 20 10:04:25 2020
krow:  5130 out of  17280  (-36.56 to -33.75) Thu Feb 20 10:04:25 2020
krow:  5400 out of  17280  (-33.75 to -30.94) Thu Feb 20 10:04:25 2020
krow:  5670 out of  17280  (-30.94 to -28.12) Thu Feb 20 10:04:25 2020
krow:  5940 out of  17280  (-28.12 to -25.31) Thu Feb 20 10:04:25 2020
krow:  6210 out of  17280  (-25.31 to -22.50) Thu Feb 20 10:04:25 2020
krow:  6480 out of  17280  (-22.50 to -19.69) Thu Feb 20 10:04:25 2020
krow:  6750 out of  17280  (-19.69 to -16.88) Thu Feb 20 10:04:25 2020
krow:  7020 out of  17280  (-16.88 to -14.06) Thu Feb 20 10:04:25 2020
krow:  7290 out of  17280  (-14.06 to -11.25) Thu Feb 20 10:04:25 2020
krow:  7560 out of  17280  (-11.25 to  -8.44) Thu Feb 20 10:04:25 2020
krow:  7830 out of  17280  ( -8.44 to  -5.62) Thu Feb 20 10:04:25 2020
krow:  8100 out of  17280  ( -5.62 to  -2.81) Thu Feb 20 10:04:25 2020
krow:  8370 out of  17280  ( -2.81 to   0.00) Thu Feb 20 10:04:25 2020
krow:  8640 out of  17280  (  0.00 to   2.81) Thu Feb 20 10:04:25 2020
krow:  8910 out of  17280  (  2.81 to   5.62) Thu Feb 20 10:04:25 2020
krow:  9180 out of  17280  (  5.62 to   8.44) Thu Feb 20 10:04:25 2020
krow:  9450 out of  17280  (  8.44 to  11.25) Thu Feb 20 10:04:25 2020
krow:  9720 out of  17280  ( 11.25 to  14.06) Thu Feb 20 10:04:25 2020
krow:  9990 out of  17280  ( 14.06 to  16.88) Thu Feb 20 10:04:25 2020
krow: 10260 out of  17280  ( 16.88 to  19.69) Thu Feb 20 10:04:25 2020
krow: 10530 out of  17280  ( 19.69 to  22.50) Thu Feb 20 10:04:29 2020
# bins in row group : 8705063
# filled bins       :    7471
krow: 10800 out of  17280  ( 22.50 to  25.31) Thu Feb 20 10:04:32 2020
# bins in row group : 8529817
# filled bins       :  134091
krow: 11070 out of  17280  ( 25.31 to  28.12) Thu Feb 20 10:04:37 2020
# bins in row group : 8334016
# filled bins       :  291162
krow: 11340 out of  17280  ( 28.12 to  30.94) Thu Feb 20 10:04:42 2020
# bins in row group : 8118129
# filled bins       :  177905
krow: 11610 out of  17280  ( 30.94 to  33.75) Thu Feb 20 10:04:46 2020
# bins in row group : 7882707
# filled bins       :   43501
krow: 11880 out of  17280  ( 33.75 to  36.56) Thu Feb 20 10:04:48 2020
# bins in row group : 7628277
# filled bins       :    5710
krow: 12150 out of  17280  ( 36.56 to  39.38) Thu Feb 20 10:04:50 2020
krow: 12420 out of  17280  ( 39.38 to  42.19) Thu Feb 20 10:04:51 2020
krow: 12690 out of  17280  ( 42.19 to  45.00) Thu Feb 20 10:04:51 2020
krow: 12960 out of  17280  ( 45.00 to  47.81) Thu Feb 20 10:04:52 2020
krow: 13230 out of  17280  ( 47.81 to  50.62) Thu Feb 20 10:04:52 2020
krow: 13500 out of  17280  ( 50.62 to  53.44) Thu Feb 20 10:04:52 2020
krow: 13770 out of  17280  ( 53.44 to  56.25) Thu Feb 20 10:04:52 2020
krow: 14040 out of  17280  ( 56.25 to  59.06) Thu Feb 20 10:04:52 2020
krow: 14310 out of  17280  ( 59.06 to  61.88) Thu Feb 20 10:04:52 2020
krow: 14580 out of  17280  ( 61.88 to  64.69) Thu Feb 20 10:04:52 2020
krow: 14850 out of  17280  ( 64.69 to  67.50) Thu Feb 20 10:04:52 2020
krow: 15120 out of  17280  ( 67.50 to  70.31) Thu Feb 20 10:04:52 2020
krow: 15390 out of  17280  ( 70.31 to  73.12) Thu Feb 20 10:04:52 2020
krow: 15660 out of  17280  ( 73.12 to  75.94) Thu Feb 20 10:04:52 2020
krow: 15930 out of  17280  ( 75.94 to  78.75) Thu Feb 20 10:04:52 2020
krow: 16200 out of  17280  ( 78.75 to  81.56) Thu Feb 20 10:04:52 2020
krow: 16470 out of  17280  ( 81.56 to  84.38) Thu Feb 20 10:04:52 2020
krow: 16740 out of  17280  ( 84.38 to  87.19) Thu Feb 20 10:04:52 2020
krow: 17010 out of  17280  ( 87.19 to  90.00) Thu Feb 20 10:04:52 2020
krow:17280 Thu Feb 20 10:04:52 2020
total_filled_bins: 659840
Writing Global Attributes
Freeing Dynamic Memory
Freeing L2 arrays

Thanks
Eligio