Please verify meaning of "Data Bins" global attribute in mapped L3 file

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
blesht
Posts: 86
Joined: Mon Sep 19, 2005 3:06 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by blesht » Mon May 01, 2017 11:52 am America/New_York

Hi - I've run into something strange (or it may just seem strange to me because I don't understand something fundamental).  I'm running though a series of mapped L3 files and I want to screen them for further analysis by testing the fraction of water pixels in the image that have data.  To do this I use a land mask generated for the same mapped region to determine the total number of water pixels in the mapped grid, and then have been dividing the individual images' "Data Bins" value by that number.   I've been finding some cases where that fraction is greater than 1, implying that there are more "Data Bins" than there are water pixels in the mapped grid.   I've been assuming that "Data Bins" represents the number of mapped pixels in the image that have valid data (not masked or flagged) - is that not correct?   Thanks, Barry

Tags:

OB SeaDAS - knowles
Subject Matter Expert
Subject Matter Expert
Posts: 269
Joined: Mon Apr 07, 2008 4:40 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by OB SeaDAS - knowles » Mon May 01, 2017 12:40 pm America/New_York

The metadata of a mapped L3 file, "data_bins", is (I believe), the valid bin count that went into creating the mapped L3 file.  However, a mapped L3 file does not have the same map projection as that of a binned file.  Binned files are equal Earth surface area bins in a sinusoidal projection.  Mapped files use an SMI projection where each pixel does NOT have the same Earth surface area.  When a mapped file is created, a mapped water pixel near the equator might derive from an aggregation of one or just a couple bins, whereas a mapped pixel far to the north could be an aggregation of far more bins.  There is not a one-to-one correspondence between a single mapped pixel and a single bin, they are not the same thing.

If you want to work with mapped data outside the bounds of just displaying imagery, then you will want to reproject it to some appropriate equal area projection such as Lambert Equal Area, etc.

Others are welcome to chime in if I don't have this explained correctly.

Danny

blesht
Posts: 86
Joined: Mon Sep 19, 2005 3:06 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by blesht » Mon May 01, 2017 12:52 pm America/New_York

Hi Danny - Thank you!   That makes sense and would explain what I see.   What I didn't understand (after all these years) was the "Data Bins" refers to the number of bins from the original global L3 file that were used to produce the mapped image rather than the number of pixels in the mapped image that had valid data.   I was hoping to avoid opening every image to do the screening (as opposed to just testing a vector with (what I thought was) the percentage of good data in all the images) but it probably won't mean all that much of a loss in time.   Again, I appreciate your help with this.    Barry

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 257
Joined: Thu Mar 05, 2009 10:25 am America/New_York
Answers: 0
Been thanked: 2 times

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by OB SeaDAS - dshea » Mon May 01, 2017 12:57 pm America/New_York

Just looked at the code for l3mapgen (assuming that is the program you use to make mapped files) and you are correct. The "Data Bins" global attribute is the number of pixels in the mapped file that have valid data.  I was worried the the metadata was just copied from the bin file, but that is not the case.  How did you generate the land mask?  Put an L3 bin file that shows the problem and I'll look into it.

don

blesht
Posts: 86
Joined: Mon Sep 19, 2005 3:06 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by blesht » Mon May 01, 2017 1:13 pm America/New_York

Darn, I was very happy with the original answer.   In fact (being old fashioned), I use smigen and not l3mapgen, I can check the source for that, though I doubt the metadata definition would have changed.   I generated the land mask using a suggestion I got some time ago from from Sean (I think).   I open and display one of the L3-mapped files, add the landmask, then output the graphics as an hdf SD.  The output file has 0s in the water pixels and 1s (if I use red for the landmask) in the land pixels.

I've found many files that have this 'problem' so it's not just one - I'll attach the one with the largest discrepancy.  I think I need to post then edit to do that (if I remember correctly).

Barry

blesht
Posts: 86
Joined: Mon Sep 19, 2005 3:06 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by blesht » Mon May 01, 2017 1:18 pm America/New_York

I tried to attache the file, but I'm not sure I was successful.   I went to the post, clicked on "Attachment," chose the file, clicked on "Upload" but nothing seems to happen.  Have I got that right?    B.

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 257
Joined: Thu Mar 05, 2009 10:25 am America/New_York
Answers: 0
Been thanked: 2 times

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by OB SeaDAS - dshea » Mon May 01, 2017 1:26 pm America/New_York

The bin file is probably too big.  You can put it on Dropbox and give me a link.

OB SeaDAS - knowles
Subject Matter Expert
Subject Matter Expert
Posts: 269
Joined: Mon Apr 07, 2008 4:40 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by OB SeaDAS - knowles » Mon May 01, 2017 1:29 pm America/New_York

Barry,

I was also quite happy with the original answer, though now we do know what data_bins actually means.  Still I believe the original answer explains a lot about this.

I'll try a follow-up answer.  Bins near land but not land themselves, may be making it in to mapped pixels of effectively lower resolution.  The mapped pixels could be centered over land, and hence be determined as land, but contain data which came from  subpixel level source bins.

This new answer is good, at least until it's not :{

Danny

blesht
Posts: 86
Joined: Mon Sep 19, 2005 3:06 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by blesht » Mon May 01, 2017 1:34 pm America/New_York

Hmmm.  Only 161Kb  - would that be too big?   I'll need to figure out Dropbox.   Difficult dealing with an old guy!   B.

blesht
Posts: 86
Joined: Mon Sep 19, 2005 3:06 pm America/New_York
Answers: 0

Please verify meaning of "Data Bins" global attribute in mapped L3 file

by blesht » Mon May 01, 2017 1:52 pm America/New_York


Post Reply