Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

Please enter here to ask a question about any NASA Science related topics!
Post Reply
marcsandoval
Posts: 15
Joined: Wed Sep 19, 2018 11:22 am America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by marcsandoval » Wed Sep 19, 2018 12:30 pm America/New_York

Hello there!

My question is basically from SeaDAS, so you can only see my SeaDAS configuration and the questions below. DINEOF is only for interpolation and therefore Figure 3.

I am trying to find a good configuration for a multivariate interpolation of SST and Chlorophyll-a in a group of islands in the South East Pacific. It is called the Juan Fernández Archipelago. I downloaded 1km L2 images of Modis-Aqua from the OceanColorWeb. The first challenge was to process them with SeaDAS to convert them to L3m format. As I read in the forum, the problem of MODIS is that it is nominally 1km at nadir, but it is not like that during all the scan, so you should work with a lower resolution. So I tried two configurations of Seadas, both with L2bin + L3mapgen: Working with resolution of 2km (170x275) and 1km (340x551) with the parameter "fudge" = 2 and 10. However, there is not much information about the consequences of use a very large number of "fudge" and in the forums I have seen it used in a range of 0 to 2 (but you can put any number: 1.5, 6, 10, etc.). As it is a very remote Island, there are not many data with which to compare, so I tried to make my images look like Figure 2 and 3 (attached) of the following paper, where they also use the same images and DINEOF:

Andrade, I., Sangrà, P., Hormazabal, S., & Correa-Ramirez, M. (2014). Island mass effect in the Juan Fernandez Archipelago (33 S), southeastern Pacific. Deep Sea Research Part I: Oceanographic Research Papers, 84, 86-99.

My configuration of SeaDAS is as follows (I will show only that of Chlorophyll-a):

For l2bin:

resolve = "2" # and then "1"
suite = "CHL"
flaguse = "ATMFAIL, LAND, HILT, HISATZEN, STRAYLIGHT, CLDICE, COCCOLITH, LOWLW, CHLWARN, CHLFAIL, NAVWARN, MAXAERITER, ATMWARN, HISOLZEN, NAVFAIL, FILTER, HIGLINT"
l3bprod = "chlor_a, chl_ocx"
prodtype = "regional" # if there is more than one image per day or "day" if not.
qual_max = "2"
oformat = "2"
night = "0"
average = "standard"
verbose = "1"
minobs = "0"
deflate = "4"

For l3mapgen

product = "chlor_a"
projection = "smi"
resolution = "2km" # and then "1km"
interp = "area"
north = "- 31.99"
south = "- 35.54"
east = "- 76.91"
west = "- 82.65"
oformat = "2"
deflate = "4"
product_rgb = "rhos_645, rhos_555, rhos_469"
fudge = "1.0" #and then 2 or 10
threshold = "0"
apply_pal = "1"
use_quality = "no"
use_rgb = "no"
suite = "CHL"
oformat2 = "png"
trimNSEW = "yes"
quiet = "false"
mask_land = "no"
land = "$ OCDATAROOT / common / landmask_GMT15ARC.nc"

In the figures attached (Figure 2), it is clear that the 2km version looks much smoother and less noisy. The difference between increasing the fudge for the 1km images is that more structures appear and the contours of the plot are "completed". Now, in comparison with Figure 2 of Andrade et al., 2014 there is a kind of similarity between the figures. However, it seems that in Andrade et al., 2014 they are less "conservative" or more "flexible" since they have much more data in each scene.

Then, I applied DINEOF. Since this is not a DINEOF forum, I will only say that it was a multivariate interpolation of SST + Chlorophyll-a and that the only parameters that I changed were: numit = 6, nev = 12 and ncv = 21

1) And the resulting figures are attached (Figure 3). Now, looking at the different Figures 2 and 3 my question is: the difference I experience between my figures and those of the paper will be due to some problem in SeaDAS (which I believe) or to the configuration of DINEOF. In the paper they didn't occupied SeaDAS, that is one of the problems.

2) About the problem of the resolution. If I resigned and I have to work with 2 kilometers, is the correct thing to do l2bin with "resolve" = 1 and then l3mapgen with "resolution" = 2km, or "resolve" = 2 and "resolution" = 2km ?. And in the opposite case, if I want to "force" 1 kilometer, what happens if you put "resolve" = 2 and "resolution" = 1km ?. What happens is that I have seen contradictory answers in the forum.

Thank you!







Tags:

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1172
Joined: Wed Sep 18, 2019 6:15 pm America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by OB.DAAC - SeanBailey » Sat Sep 22, 2018 4:37 pm America/New_York

If I read your post correctly, I see that your question boils down to wanting to know how best to bin and map data for your purpose.

The l2bin program (currently) treats each L2 pixel as a point in space based on the L2 pixel center (the lon/lat values in the L2 file are for pixel center).
It puts the pixel in the nearest L3 bin (also based on bin center).  If you bin to 1km resolution, near NADIR for the L2 swath, the nominally 1km
L2 pixels almost align with the bin contiguously, but as you move out on the scan, the pixel centers get farther apart than 1km, so there will
be gaps in the L3 product. 

By default,  mapper (l3mapgen) takes each mapped pixel and looks up in the bin file the closest bin to the mapped pixel center, so mapping a 1km
bin file of a nominally 1km sensor to a 1km map resolution will produce some pretty Moire patterns.  The mapper has two features to help mitigate this:
fudge and interp.

The fudge factor, is just that.  It "fudges" the size of the area to look up in the bin file by the corresponding factor provided.  When the mapper looks up
a location in the bin file and does not find a match, you get missing data in the map.  With the fudge factor, the mapper will look up a larger region in the
bin file to associate with the map pixel.  The current version only does this look up if no retrieval was found using the defined resolution. 

The interp option allows you to select 'area' (the default is nearest-neighbor, so no interpolation).  With area interpolation, the mapper uses the corners of
the mapped pixel to define an area to search in the bin file.  This is a useful feature, since the projection of the map is not going to line up with the projection
of the bin file.

The resolution of the map is affected by both the resolution chosen for binning and the resolution chosen for mapping.
While you can generate a 1km map from a 2km bin file, it's really just a super-sampled 2km map..you can do it, but I wouldn't call it a 1km map.
It may make a smoother output image, if that's the goal.

I hope this helps in your decision making
Sean

marcsandoval
Posts: 15
Joined: Wed Sep 19, 2018 11:22 am America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by marcsandoval » Mon Sep 24, 2018 12:04 pm America/New_York

Dear Sean,

Thank you very much for your answer. I don't know much about the subject or the concepts and it helped me a lot.
Following what you tell me, and to see if I understood correctly, each combination will give the following:
a) resolve = "1" (l2bin) and resolution = "1km" (l3mapgen). I will keep data of 1km of resolution but I will have to play with 'fudge' to eliminate noise until I find images that make me feel calm. In which case, what should be the maximum value of 'fudge' to try?
b) resolve = "2" (l2bin) and resolution = "1km" (l3mapgen). I will create 1km resolution data through an upsampling of a 2km bin file. It will give a more smoothed images but they may not really be real data.
c) resolve = "1" (l2bin) and resolution = "2km" (l3mapgen). I will go down to 2km of resolution and I will keep the gaps product of bin at 1km a nominally 1km sensor file.
d) resolve = "2" (l2bin) and resolution = "2km" (l3mapgen). I will go down to 2km of resolution and those gaps will disappear, as l2bin and l3mapgen search and order the 1km pixels in 2km "boxes".

Now, if what interests me is having reliable (true) data of the highest possible resolution instead of more smooth images, what do you advise me to choose? In this case, the "a" option does not look so bad, since it maintains resolution and if it does not find pixels at 1km, it will simply leave it as a gap (it will not create data). Or the option "d" since watch from a more "thick" point of view ends up being more "reliable" ?.

Finally, I'm confused a little with the "b" option, especially when watching The OCSSW Cookbook:

https://seadas.gsfc.nasa.gov/help/OcsswCookbook/ocsswCookbookMappedSst.html

Because for all options, including the "Science Quality", they take up resolve = "1" and resolution = "750". Only 'qual_max' and 'flaguse' differentiate the other qualities. In the "Notes" seem to say that it is a usual practice bin in low resolution and map at high.

Regards!

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1172
Joined: Wed Sep 18, 2019 6:15 pm America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by OB.DAAC - SeanBailey » Mon Sep 24, 2018 3:30 pm America/New_York

> what should be the maximum value of 'fudge' to try?


The larger the fudge you use the more "blocky" the data will appear where there was missing data that is now filled with adjacent bin data.
Personally, I'd try to keep it under 2, but I'd also only use it for making pretty pictures.

> Now, if what interests me is having reliable (true) data of the highest possible resolution instead of more smooth images, what do you advise me to choose?


Either a) or d) without  the fudge option, but with the interp=area option. 

> Finally, I'm confused a little with the "b" option, especially when watching The OCSSW Cookbook


The  l2bin program has a finite set of possible bin resolutions.  VIIRS ocean color and SST products have a native NADIR resolution of 750m, which is not
possible for the binner to match.  The closest bin resolution (for the 32bit binner code used in the Cookbook example) is resolve=1 (which is really more like 1.15km, as 2 is really 2.3 and 4 is really 4.6 and 9 is really 9.2....) So when Danny wrote the cookbook he chose to bin at resolve=1 and map to 750m ...it's not that far off ;)

Sean

OB SeaDAS - knowles
Subject Matter Expert
Subject Matter Expert
Posts: 221
Joined: Mon Apr 07, 2008 4:40 pm America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by OB SeaDAS - knowles » Tue Sep 25, 2018 9:22 am America/New_York

I'm glad this cookbook example is useful.

Note: besides illustrating how to use l2bin and l3mapgen, the point of this cookbook example was to illustrate some data quality and data retrieval options.  The bin and map resolutions were not adjusted between the 4 cookbook cases so as to not confuse the quality issue with other parameters being simultaneously altered.

I'm not so sure that the notes "seem to say that it is a usual practice bin in low resolution and map at high".   Also note that the Earth surface spatial footprint of the bins and the plate caree mapped image pixels are NOT the same even if you set them both to "1" or both to "2".  Choosing a slightly higher resolution for the mapped image has not only the advantage a smoother picture but enables an overlay of a higher resolution Land mask. 

Really, plate caree mapping is a picture (be it a pretty one or an ugly one) and care should be used when calling it's individual pixels "reliable (true) data".

For statistics, if you cannot work with the binned data, then I would suggest remapping the l3mapgen output file(s) using the Reproject or Mosaic tools and picking an equal-area projection such as Sinusoidal or Lambert Azimuthal Equal Area.

Danny

marcsandoval
Posts: 15
Joined: Wed Sep 19, 2018 11:22 am America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by marcsandoval » Wed Sep 26, 2018 10:27 am America/New_York

Thank you very much Sean and Danny, your answers help me a lot.

Danny, I work with Matlab to perform statistical analysis and as you say, I can't work with binned data since I need regular matrices.
Taking the recommendations of Sean and because I need high resolution (the islands are very small) and good data quality (scientific quality), I thought I would use resolve = "1" and resolution = "1km", in addition to the other parameters which I defined in my configuration above. That another step that you tell me l2bin + l3mapgen + Reproject, what would it do? Would it be to work in other software such as Arcgis for example (which is not my case)?

Regards!

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1172
Joined: Wed Sep 18, 2019 6:15 pm America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by OB.DAAC - SeanBailey » Sat Sep 29, 2018 2:26 pm America/New_York

The l3mapgen program reprojects the bin file.  The output is limited to a few datatypes.  The SeaDAS Reproject tool can output more formats.  Reprojecting a reprojected file always runs the risk of under or oversampling the data.  If you don't need any feature the Reproject tool has that isn't in l3mapgen, no need to use it.

Sean

srdasara
Posts: 1
Joined: Sun Mar 22, 2020 7:25 am America/New_York

Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by srdasara » Wed Mar 25, 2020 10:05 pm America/New_York

Hi Marco,

This is a reply to a post you made over a year ago, apologies, but I'd really appreciate any help on the topic!

You mentioned that you used DINEOF through MATLAB... could you please let me know how you got this working?? Any help would be much appreciated. I definitely have a fair bit of experience with MATLAB now and I'm not sure why this isn't working for me. I noticed the MATLAB wrapper on the DINEOF main webpage, here: https://svn.oss.deltares.nl/repos/openearthtools/trunk/matlab/applications/+dineof/

However, the example code (i.e., tst.m) simply isn't running for me, and in fact points to a function that doesn't exist through the DINEOF installation (dineof.run -- or am I mistaken?) I'm not sure if the wrapper is outdated? Or perhaps I'm not understanding something with the Matlab installation?

Is there a specific way you are running your chlorophyll data matrix through DINEOF? Do you have a coding sample, for example, that you would be willing to share? I've been stuck on this for quite a while now and any pointers would be a great help!

Cheers!

gnwiii
Posts: 606
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 1

Re: Chla+SST 1km Modis-Aqua L2 to L3m (SeaDAS 7.5.1) and L4 (DINEOF)

by gnwiii » Wed Apr 07, 2021 8:10 am America/New_York

marcsandoval wrote: Wed Sep 26, 2018 10:27 am America/New_York Thank you very much Sean and Danny, your answers help me a lot.Danny, I work with Matlab to perform statistical analysis and as you say, I can't work with binned data since I need regular matrices.Taking the recommendations of Sean and because I need high resolution (the islands are very small) and good data quality (scientific quality), I thought I would use resolve = "1" and resolution = "1km", in addition to the other parameters which I defined in my configuration above. That another step that you tell me l2bin + l3mapgen + Reproject, what would it do? Would it be to work in other software such as Arcgis for example (which is not my case)?Regards!
If your statistics are things like comparing satellite measurements with in situ measurements, you can use "l3bindump" to obtain bin values and locations for specified locations or regions (circle or square in lon-lat coords). I use R for this, but Matlab and R have considerable overlap in capabilities. For regions with complicated shapes you can use l3bindump for a circle or box, then select bins whose centers lie inside some polygon.

I haven't used ArcGIS recently, but many colleagues do use it. Current versions seem to have good support for NetCDF4-CF files such as you get from l3mapgen. Current versions of QGIS also have good support (via GDAL libraries) for NetCDF4-CF files.

Post Reply