l2gen error that references ...seadas-viirs-v2.0/seadas/build/src/l2gen/prodgen.c

[blesht]

Hi - I'm reprocessing some SeaWiFS files from L1 to L2 using l2gen and have run into an error I've not seen before.  I should note that I'm doing the reprocessing because I needed to add some ancillary variables (ntilts, etc.) I need for some downstream calculations that no longer are included in the default suite.  The error I'm getting shows up in the last line of the output excerpt below:

Morel f/Q table from file /Applications/seadas-7.4/ocssw/run/data/common/morel_fq.nc

No Raman scattering correction calculated for Rrs.

Loading climatology file /Applications/seadas-7.4/ocssw/run/data/common/alpha510_climatology.hdf
Loading climatology file /Applications/seadas-7.4/ocssw/run/data/common/taua865_climatology.hdf
Loading aerosol properties for PAR from /Applications/seadas-7.4/ocssw/run/data/seawifs/aerosol/seawifs_aerosol_par.dat.
-E- /Users/seadas/seadas-viirs-v2.0/seadas/build/src/l2gen/prodgen.c Line 852: Unknown product catalogue ID -1.

Any idea about what is going on?

[blesht]

Update - I removed the variables I had added to the product list and the script now runs the way it used to (which is to say correctly).  The question now is how do I add the following to the L2 file:

ntilts, tilt, tilt_flags, and tilt_ranges.

[gnwiii]

Do all the added ancillary variables have entries in $OCSSWROOT/run/data/common/product.xml?   It might help to give the full l2gen command or at least a list of the added variables.  You could try creating level-2 files for one variable at a time to see if the issue is associated with a particular variable.

I see you provided the list of 4 variables, all of whcih do have entries in product.xml.

[blesht]

Further update:  Rather than run my script I used the GUI to try and create an L2 file that included ntilts, tilt_flags, and tilt_ranges.  That too failed with the original error.  I need those variables for some downstream processing.  How do I get them?

I'm currently using SeaDAS 7.4 running on a MacbookPro under MacOSX 10.12.6.

[blesht]

Hi George.  Thanks, this is the line in my script that includes the 'extra' variables.

echo l2prod=Rrs_412 Rrs_443 Rrs_490 Rrs_510 Rrs_555 Rrs_670 chlor_a Kd_490 par ntilts tilt tilt_flags tilt_ranges l2_flags >> .parin

As I noted, I removed all the tilt-related variables and the script runs successfully.  I tried adding only ntilts and it fails.  The fact that I get the same failure and error when I run directly from the GUI suggests that my script is not at fault.   I did check my products.xml file and it also has the products listed as you found.  They do not, however, seem to appear in $OCSSWROOT/build/src/l2gen/prodgen.c

Barry

[OB SeaDAS - dshea]

tilt is in every L2 file.  On the other hand ntilts, tilt_flags, and tilt_ranges have never been written to NetCDF L2 files and have not been populated in HDF L2 files for over 10 years.  They are added as a VGroup, but never filled in as far as I can tell.  I stopped looking backwards in our code repository when l2gen was called MSL12.  Looks like you are going to have to pull these parameters out of the L1 file.

don

[blesht]

Hi Don - thanks for checking.  I have some older L2 files generated by using l2gen v6.5.5 that include ntilt, tilt_flags, and tilt_ranges by default.  Here is an excerpt from the netCDF header dump from one of them:

        long ntilts(unit) ;
                ntilts:long_name = "Number of scene tilt states" ;
        short tilt_flags(mtilt) ;
                tilt_flags:long_name = "Tilt indicators" ;
                tilt_flags:valid_range = -1s, 3s ;
        short tilt_ranges(mtilt, ltilt) ;
                tilt_ranges:long_name = "Scan-line number ranges of scene tilt states" ;

// global attributes:
                :Title = "SeaWiFS Level-2 Data" ;
                :Sensor Name = "SeaWiFS" ;
                :Product Name = "S2010078205641.L2.x.hdf" ;
                :Software Name = "l2gen" ;
                :Software Version = "6.5.5" ;

So it was part of l2gen at some point in the not (too terribly distant) past.  

Perhaps I'm last SeaDAS user who every once in a while has to go back to a script that uses some of the SeaWiFS6.4 commands to accomplish something that I haven't been able to do in SeaDAS7.  The script I'm using invokes the load command which, in turn, tries to get values for ntilts, tilt_flags, and tilt_ranges.  Perhaps I can find a way to get around this, but it really is frustrating when something that used to work fine breaks because of a software update.

[gnwiii]

In $OCSSWROOT/build/src/l2gen/l2_gneric.c:

[...]
    /* Do we have the extra meta-data for SeaWiFS */
    int seawifs_meta = 0;
    if (l2file->sensorID == SEAWIFS) {
        int32 sds_id;
        if ( sd_select(sd_id,"scan_ell",&sds_id) == 0 )
            seawifs_meta = 1;
    }
[...]
    if (seawifs_meta) {

        /* Sensor Tilt */
        DPTB( v_attach(h_id, &v_id)      );
        Vsetclass(v_id, "Per File Data");
        Vsetname(v_id, "Sensor Tilt");
        DPTB( AddSdsToVgroup(sd_id, v_id, "ntilts")    );
        DPTB( AddSdsToVgroup(sd_id, v_id, "tilt_flags")  );
        DPTB( AddSdsToVgroup(sd_id, v_id, "tilt_ranges")  );
        Vdetach(v_id);

    }


If you are able to build l2gen from source it might be informative to add a few print statements to see if seawifs_meta is being set to 1.

[blesht]

Thanks again, George.  Do I have it right that the sd_select line tests for the existence of "scan_ell" in the L1A file and sets seawifs_meta to 1 if the test is passed?  I have checked the L1A files and "scan_ell" is there.  Would there have to be some code that even if ntilts etc. were added to the Vgroup, that Vgroup would be written to the output file?  I couldn't find anything like that in l2_generic.c.  Anyway, the fix I had in mind was to write some code that would open the existing L2 file(s) in write/edit mode, find the corresponding L1A file(s) and grab the missing information, then write that information to the L2 file(s) before saving it (them).   I'd really like to avoid completley regenerating all those L2 files with a modified l2gen if I can avoid doing so.

Barry

[gnwiii]

Adding the missing bands to an existing L2 file would be faster than running an updated l2gen.  In general, it is probably better to minimize use of trivial algorithms in  l2gen and do more trivial processing separately as you propose. Tracking down what looks like a glitch in l2gen could, however, be a useful exercise for learning more about l2gen internals (such knowledge is useful when hacking l2gen to explore alternate, non-trivial, algorithms).