Please verify meaning of "Data Bins" global attribute in mapped L3 file

[blesht]

Hi - I've run into something strange (or it may just seem strange to me because I don't understand something fundamental).  I'm running though a series of mapped L3 files and I want to screen them for further analysis by testing the fraction of water pixels in the image that have data.  To do this I use a land mask generated for the same mapped region to determine the total number of water pixels in the mapped grid, and then have been dividing the individual images' "Data Bins" value by that number.   I've been finding some cases where that fraction is greater than 1, implying that there are more "Data Bins" than there are water pixels in the mapped grid.   I've been assuming that "Data Bins" represents the number of mapped pixels in the image that have valid data (not masked or flagged) - is that not correct?   Thanks, Barry

[OB SeaDAS - knowles]

The metadata of a mapped L3 file, "data_bins", is (I believe), the valid bin count that went into creating the mapped L3 file.  However, a mapped L3 file does not have the same map projection as that of a binned file.  Binned files are equal Earth surface area bins in a sinusoidal projection.  Mapped files use an SMI projection where each pixel does NOT have the same Earth surface area.  When a mapped file is created, a mapped water pixel near the equator might derive from an aggregation of one or just a couple bins, whereas a mapped pixel far to the north could be an aggregation of far more bins.  There is not a one-to-one correspondence between a single mapped pixel and a single bin, they are not the same thing.

If you want to work with mapped data outside the bounds of just displaying imagery, then you will want to reproject it to some appropriate equal area projection such as Lambert Equal Area, etc.

Others are welcome to chime in if I don't have this explained correctly.

Danny

[blesht]

Hi Danny - Thank you!   That makes sense and would explain what I see.   What I didn't understand (after all these years) was the "Data Bins" refers to the number of bins from the original global L3 file that were used to produce the mapped image rather than the number of pixels in the mapped image that had valid data.   I was hoping to avoid opening every image to do the screening (as opposed to just testing a vector with (what I thought was) the percentage of good data in all the images) but it probably won't mean all that much of a loss in time.   Again, I appreciate your help with this.    Barry

[OB SeaDAS - dshea]

Just looked at the code for l3mapgen (assuming that is the program you use to make mapped files) and you are correct. The "Data Bins" global attribute is the number of pixels in the mapped file that have valid data.  I was worried the the metadata was just copied from the bin file, but that is not the case.  How did you generate the land mask?  Put an L3 bin file that shows the problem and I'll look into it.

don

[blesht]

Darn, I was very happy with the original answer.   In fact (being old fashioned), I use smigen and not l3mapgen, I can check the source for that, though I doubt the metadata definition would have changed.   I generated the land mask using a suggestion I got some time ago from from Sean (I think).   I open and display one of the L3-mapped files, add the landmask, then output the graphics as an hdf SD.  The output file has 0s in the water pixels and 1s (if I use red for the landmask) in the land pixels.

I've found many files that have this 'problem' so it's not just one - I'll attach the one with the largest discrepancy.  I think I need to post then edit to do that (if I remember correctly).

Barry

[blesht]

I tried to attache the file, but I'm not sure I was successful.   I went to the post, clicked on "Attachment," chose the file, clicked on "Upload" but nothing seems to happen.  Have I got that right?    B.

[OB SeaDAS - dshea]

The bin file is probably too big.  You can put it on Dropbox and give me a link.

[OB SeaDAS - knowles]

Barry,

I was also quite happy with the original answer, though now we do know what data_bins actually means.  Still I believe the original answer explains a lot about this.

I'll try a follow-up answer.  Bins near land but not land themselves, may be making it in to mapped pixels of effectively lower resolution.  The mapped pixels could be centered over land, and hence be determined as land, but contain data which came from  subpixel level source bins.

This new answer is good, at least until it's not :{

Danny

[blesht]

Hmmm.  Only 161Kb  - would that be too big?   I'll need to figure out Dropbox.   Difficult dealing with an old guy!   B.

[blesht]

Hi Don - let's if this works:   https://www.dropbox.com/s/th74n2qz1we59bw/S1998238183312.chlor_a.smimap.L3.hdf?dl=0

Thanks, Barry