How to calculate percentile amount of a range chl value

[antu]

Hello experts,
I am dealing with level-3 mapped monthly average chlorophyll value. I want to find the percentage of a range vlue in the study area. Let me explain, There are diverse amount of value exist in the image. Assume that I make some ranges of chlorophyll concentration like (0.1-0.99) , (1-1.99), (2-4.99), (5-20) mgm^-3. Now if I want to find the percentile value for (1-1.99)mgm^-3 in the whole study area. what procedures can I follow ? Is it possible to calculate how much percentiles of pixels bears (1-1.99)mgm^-3 range value for a specific study domain ?

Thanks,
Antu

[OB SeaDAS - knowles]

Hi,

Statistics, regions, masking, and mapped files is not a subject which can be answered in brief.   The next release of SeaDAS will contain a substantially revised Statistics GUI and much more extensive documentation on the subject.  In the meantime, since I cannot yet refer you to that documentation, here's a long answer to your question.

First, it all depends on what you mean by percentile.  SeaDAS has a Statistics Tool with a percentile plot and associated percentile thresholds 80%, 85% etc.  These percent threshold fields are based on binning and you need to set an appropriate binning resolution (currently with the accuracy parameter).  When using these fields,  NaN (null) values are ignored and the results give you a value based only on valid sample data.

I would suggest exploring this Statistics Tool in association with the Mask Manager.  You can define masks on the chlor_a ranges, one mask per range.  When you run the Statistics Tool (from the GUI) you can select these masks and get statistics results for each mask criteria.  You can get the total number of pixels which meets your masking criteria.  GPT command line version of the tool can also be used with the exception that you will use the valid pixel expression field for your masking. 

You can define a specific study domain (region of interest)  for instance by drawing a polygon on the image, or importing a shapefile, which will automatically get an associated mask.  By combining the region mask and the chlor_a range masking criteria into a single mask (one mask per chlor_a range)  you can then use the GUI statistics.  For GPT command line you would use the shapefile option to define your region.

Of importance and note is that the level-3 mapped images have a smi plate caree projection (not equal Earth surface area per pixel).  Since your question is of a statistical nature, you should consider using a tool such as the Reproject Tool and reprojecting your data to a Sinusoidal or other equal area projection and doing statistics on that file.

If you are trying to find the total number of pixels in a given region, which include all pixels, you can use the Math Band tool to create a band where all pixels have data.  For instance create a band with expression=1.  Then all pixels in the new band get assigned 1 as the value, with no NaN values.  Then you can do Statistics on this band to get total pixel counts for the masking and region criteria mentioned above.

Danny

[gnwiii]

There is already very good advice from Danny Knowles, but I wanted to mention another approach.  If you have the OCSSW processing system you can use l3bindump with the level-3 binned data to generate ASCII files with binned data values.  You can then use the data with one of the many full-fledged statistical languages (e.g., R) to do a wide range of statiscal calculations.   Because the bins are approximately the same area, you avoid the extra remapping step to an equal area projection.  Even without the OCSSW processing systems, there is  adequate documentation for the format to allow someone familiar with one of the many programing languages that have good support for NetCDF4 format files to read data from current NASA binned files.

[OB SeaDAS - knowles]

Currently, the closest SeaDAS itself can come to doing statistics on bin files would be to take the bin file and run it through the OCSSW processor l3mapgen with the projection set to "raw", which is a raw dump of the bin file into the map file raster.  The resultant file can then be viewed and analyzed within SeaDAS, with the current limitation that there is no geographic lat/lon info for the pixels in the file.  You can then do global statistics on this file in SeaDAS, but not regional (unless you visually define a region).

[antu]

Thanks Daniel Knowles, for your elaborate answer. It works.I also appreciate you for your brief video tutorials and webinar.:smile: 
I have another problem i.e, Can you tell me about no data layer ? Why there is data missing and show NaN value of chl-a within the water area? If you give me a elaborate answer or any reading documents, that will be helpful for me. I have attached a image by which you can observe how much data is missing some of my images.

Thanks,
Antu

[antu]

Thanks gnwiii for this another approach. I did it with masks and just calculated it.
gnwiii, you are truly helpful. Whenever I faced some problems and posted on forum, you are one who always help me with helpful advice. Thanks again.

If you have any idea about no data layer and why there is data missing in the coast water ? Please let me know. I need some elaborate answer or some documents with references. That will be helpful for me.

regards,
Antu

[OB SeaDAS - knowles]

If you take a look at the individual level-2 scenes for May 2017 you will see a lot of clouds, straylight and glint, and consequently chlor_a cannot be determined.

If the value cannot be determined for a pixel then that pixel gets no data and hence is marked as no-data.  There is nothing special about the No-Data Layer.  It merely shows which pixels have no-data and enables you to set the color and visibility for these no-data pixels.

I would suggest downloading a couple level-2 files for a few days in Summer and Winter in your region and observing the flags (via the Pixel Info window or by toggling them in the Mask Manager) which are set to get a feel for what is going on.  Certainly glint is going to be an issue in this region in the Summer.

Level-3 files produced on OB.DAAC apply strong quality control masking on the data.  In order for a pixel from an individual scene to make it into a level-3 file it must fully meet this quality criteria.  Here is a mask available in SeaDAS which illustrates the level-2 flags which must be met:

Quality_L3 =
!( l2_flags.ATMFAIL or l2_flags.LAND or l2_flags.HIGLINT or l2_flags.HILT or l2_flags.STRAYLIGHT or l2_flags.CLDICE or l2_flags.COCCOLITH or l2_flags.HISOLZEN or l2_flags.LOWLW or l2_flags.CHLFAIL or l2_flags.NAVWARN or l2_flags.MAXAERITER or l2_flags.CHLWARN or l2_flags.ATMWARN or l2_flags.NAVFAIL or l2_flags.FILTER )

Note: if for any reason you want to loosen (or alter) this masking restriction, you can use SeaDAS to process the level-2 files to create your own custom level-3 files.

Danny

[antu]

Thanks knowles. I will try to follow your advice.