Please explain mk_matchup verbose exclusion message

blesht · by **blesht** » Thu Oct 15, 2020 10:02 am America/New_York

Hi. I've got my mk_matchup script running well now but, using the --verbose option, I am getting some exclusion messages. One of them,

"No matchup: failed MIN_VALID_SAT_PIX, required = 50. found = 40. Exclusion level = 4, Matrix row = 11", is pretty clear and relates to the percentage of acceptable pixels in the 5x5 box which is a user controlled parameter.

The other,

"No matchup: failed MIN_VALID_SAT_PIX, extracted satellite pixels less than box size, required = 5 found = 0. Exclusion level = 3, Matrix row = 12" , is not so clear (to me at least).

Could you please explain what conditions trigger the second message? Does it have something to do with the L2 flags that are set for that pixel? If so, can you point me to where those flags are checked? Does the triggering condition involve more than the solar and satellite zenith angles? Why isn't this just a special case covered by the first message?

Thanks!

blesht · by **blesht** » Fri Oct 23, 2020 1:38 pm America/New_York

Unusual for you guys - no answer to this query yet. Maybe it got confused with my earlier post about having trouble running mk_matchup that Sean resolved. I'd still like an answer to this question about what triggers the cited error message. Thanks.

by **OB.DAAC - SeanBailey** » Fri Oct 23, 2020 4:32 pm America/New_York

Pretty opaque to me, too ..didn't write the code :grin:

But I can read the code...and both are because the MIN_VALID_SAT_PIX you asked for was not met. The first because there were 40 pixels but you said "need 50" (I think the numbers are in percent, so 40% were valid but you asked for 50%). The second case is because the extracted box contained zero unflagged pixels which is less than the requested box size, in this case a 5x5 box was requested, so 5. Weird, I wouldn't have worded the error that way. I *think* the intent was to try to match the conditions for a valid matchup that we described in the Bailey and Werdell (2006) paper. In that we required 50% of non-land pixels to be valid, but there needed to be at least 5 pixels. I think...that was 14 years and 6 kids ago...

Sean

blesht · by **blesht** » Fri Oct 23, 2020 5:39 pm America/New_York

Hi. Thanks! Yes, the first message does relate to the percent of accepted pixels in the box. Both the percentage and the box size are parameters the user can adjust. What I wasn't clear about (and probably didn't express very clearly) where the program reads the flags that might make a pixel unacceptable. I don't do Python or Perl (but remember PL/I for what that is worth these days) and did look at the code but I couldn't find the routine in which those flags (presumably from the l2flags) were read and interpreted. I've modeled my own matching criteria after yours (B&W, 2006) and what got me wondering about this was that mk_matchup seemed to be throwing out locations (with that second error) that I was able to retrieve with out_track (using the same L2 files as input). I'll go back and check things again. If I still have questions, I'll get back to you.

14 years & 6 kids! Outstanding.

Barry

by **OB.DAAC - SeanBailey** » Sat Oct 24, 2020 8:47 am America/New_York

Barry,
The second test is just a special case of the first. mk_matchup.py runs val_extract, which is where the exclusion criteria are applied ...and the number of unflagged pixels is computed. val_extract is compiled C code, so you won't see in the l2_flags being read by the python script. In the example you showed, there were zero unflagged pixels, so the second condition is met...as well as the first

Oh, and 6 is only half...go back another 12 years and there's another 6...

Sean