Page 1 of 1

L2gen error with 3x3 straylight masking

Posted: Mon May 31, 2021 6:53 pm America/New_York
by schckngs
Hey all,

This thread (viewtopic.php?f=7&t=1875&sid=e346b30e9d ... 524e9839a8)has been inactive for a while but I found it helpful as I'm experiencing the same issue with images here and there.
I've been running OCSSW to process L1A > L2 MODISA imagery with output products including Rrs, sst, etc. I wanted to use a 3x3 straylight filter so changed that in the msl12_filter.dat file.

Now that I have processed my files over the last few weeks and am cross-checking, I'm noticing 95%+ processed just fine, but here and there this error popped up and the image wasn't processed.
Offhand one image that fails the 3x3 straylight flag is A2007180190000.L1A_LAC, while A2003206173500.L1A_LAC works with both a 3x3 or back to the original 7 wide 5 high.
This is with l2gen v 9.5.0 (V2019.3).


Re: L2gen error with 3x3 straylight masking

Posted: Tue Jun 01, 2021 2:12 pm America/New_York
by OB.DAAC - SeanBailey

The code probably isn't trying to be lazy... can you verify that the GEO filename is correct? I ask because that A2007180190000.L1A_LAC produced a GEO file like this "A2007180190001.GEO" when I ran it through modis_GEO. So, l2gen failed because my assumption that the GEO file would be named "A2007180190000.GEO". When I give l2gen the correct GEO filename, it works with a 3x3 straylight masking. (OK, works is relative. 99% of the scene is cloud/straylight)


Re: L2gen error with 3x3 straylight masking

Posted: Tue Jun 01, 2021 3:01 pm America/New_York
by schckngs
Hey Sean,

Thanks for looking. Yeah, that file does produce a GEO file with a "1"...lots of those in 2006/2007.
I re-checked and this file still fails for me with 3x3 but passes with 7x5 (accting for the "1" in the GEO file :) ).
With 3x3:

Loading SSES table from /home/hilborna/ocssw/share/modis/aqua/cal/sst_sses_modisa_v6.5.hdf

Loading SSES table from /home/hilborna/ocssw/share/modis/aqua/cal/sst4_sses_modisa_v6.5.hdf
Loading SST4 lat band coefficients from /home/hilborna/ocssw/share/modis/aqua/cal/
6 -90.00 -40.00 0.734328 1.015192 0.526194 2.064882 -0.003070 -0.000157 -0.000224
6 -40.00 -20.00 0.518013 1.022969 0.564027 2.329543 -0.003196 -0.000024 -0.000251
6 -20.00 0.00 0.240000 1.057443 0.318513 2.736735 0.009355 0.000190 -0.000179
6 0.00 20.00 0.254244 1.060247 0.309935 2.910488 0.009790 0.000264 -0.000195
6 20.00 40.00 0.352444 1.027116 0.604485 2.466619 0.005873 0.000057 -0.000278
6 40.00 60.00 0.386893 1.013935 0.700843 2.018802 -0.001995 -0.000118 -0.000284
6 60.00 90.00 0.747807 1.002158 0.663846 2.069339 -0.045267 -0.001026 -0.000340
-E- /home/seadas/ocssw/src/l2gen/sst.c line 3024: L1 queue size of 3 is too small for 3 line Bt40 average around line 0.

A copy of the straylight mod from msl12_filter.dat:
stlight, 5, 3, 3, 1
stlight, 10, 3, 3, 1

A couple other files that fail for me but pass with the regular mask are A2003236130000 and A2003254142500.
For now I think I'll reprocess the failed files with a 7x5, there'll be a small inconsistency thru the masks in the image time series but faster than reprocessing everything...

Cheers and thanks for looking,

Re: L2gen error with 3x3 straylight masking

Posted: Tue Jun 01, 2021 3:36 pm America/New_York
by OB.DAAC - SeanBailey
Ah, worked for me because I used a more recent version of the code. It should work for you as well if you update OCSSW to the latest released with SeaDAS8. With the version you're running, it should never work for you with those parameters in the filter file.

It's not the source L1 that is the issue, but using a filter file with a 3x3 kernel and sst in the product list. So, you've baffled me. Not hard to do ;)

The newer code sets the minimum queue (number of L1 scans to read in at a time) to allow SST processing to work even if the filter file doesn't require a kernel of at least 5 scans. The filter code can modify the kernel size to ensure it has enough, previously it 'worked' because while the default was 3, the filter file defined a kernel with 5. SST requires 5 for some of the spatial homogeneity tests.


Re: L2gen error with 3x3 straylight masking

Posted: Tue Jun 01, 2021 4:41 pm America/New_York
by schckngs
Thanks Sean, for next time I'll make sure SeaDAS is updated first....

I wonder if the working / not working has something to do with spatial coverage in the bands used. Regardless, thanks for looking. Cheers