l3mapgen/smigen redux

Please enter here to ask a question about any NASA Science related topics!
blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Wed Jul 15, 2020 1:57 pm America/New_York

Finally convinced that I need to switch to l3mapgen after (many, many) years of processing with smigen, I've started to take a closer look at the downstream consequences.  Many of my analysis codes and ancillary files (e.g. landmasks, coastlines, bathymetry) use pre-defined areas of interest that were based on the grids produced by smigen.  Although it appears that the output grids from smigen are identical for some of the grids produced by l3mapgen (using either the default or specifying the "smi" projection - which should be the same, I know), some are not.  For instance, my Lake Michigan smigen grid is 225x168 but the l3mapgen grid generated using the exact same lat/long limits is 226x168.  Similarly, my original (smigen) Lake Huron grid is 177x255 but the l3mapgen grid is 178x254.  The metadata for both types of output files have the same geographic parameters (e.g. north, south, east, west limits).

So first, what might account for the difference in sizes?  Is there something I can do when using l3mapgen to ensure the new grids match the old ones? 

[DITED - I noticed that for Lake Michigan, all the parameters are identical except for the "Latitude Step."   For the smigen file it is   :Latitude Step = 0.020888893f ;  and for the l3mapgen file it is :Latitude Step = 0.020796463f ;.   Now the question is why are the latitude steps different?

Second, looking through the forum for issues related to l3mapgen I see references to the "old" l2bin and the "old" l3mapgen.  I'm sure I'm using the "new" l3mapgen, but many of my l2bin files are pretty old.  Do I need to regenerate them too?

Thanks, Barry

Tags:

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 214
Joined: Thu Mar 05, 2009 10:25 am America/New_York

l3mapgen/smigen redux

by OB SeaDAS - dshea » Thu Jul 16, 2020 8:41 am America/New_York

Can you put a bin file that exhibits this problem somewhere where I can grab it.  Then send me the exact command line for smigen and l3mapgen and I will take a look at this.

don

blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Thu Jul 16, 2020 10:36 am America/New_York

Morning, Don.   Thanks.  I'll see if I can figure out some place to post the bin file so you can grab it.  New cyber-security protocols at work make it much more complicated than it used to be to share files with "outsiders."   In the mean time here are the relevant excerpts from the script I used to test smigen/l3mapgen.  I was interested in how different combinations of parameters affected the output, so I was calling smigen twice (to see how log and linear scaling might appear differently) and l3mapgen three times to look into the differences in the interpolation method.

Excerpt 1 - common parameters

resolution="2km"
precision="F"
stype1="lin"                  # linear scaling vs log (default)
stype2="log"
stat=1

Excerpt 2 - calls to smigen/l3mapgan

smigen ifile=${INFILE} prod=${2} ofile=${ofile1} resolution=${resolution} latnorth=${latnorth} latsouth=${latsouth} lonwest=${lonwest} loneast=${loneast} meas=${stat} stype=${stype1} precision=${precision}
smigen ifile=${INFILE} prod=${2} ofile=${ofile2} resolution=${resolution} latnorth=${latnorth} latsouth=${latsouth} lonwest=${lonwest} loneast=${loneast} meas=${stat} stype=${stype2} precision=${precision}
l3mapgen ifile=${INFILE} product=${2} ofile=${ofile3} resolution=${resolution} north=${latnorth} south=${latsouth} west=${lonwest} east=${loneast} oformat="hdf4"
l3mapgen ifile=${INFILE} product=${2} ofile=${ofile4} resolution=${resolution} north=${latnorth} south=${latsouth} west=${lonwest} east=${loneast} oformat="hdf4" interp="bin"
l3mapgen ifile=${INFILE} product=${2} ofile=${ofile5} resolution=${resolution} north=${latnorth} south=${latsouth} west=${lonwest} east=${loneast} oformat="hdf4" interp="area"

Barry

blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Thu Jul 16, 2020 1:28 pm America/New_York

Hi again.  Still waiting to hear from the IT people about cyber-safe way to post a file for you to look at.  In the meantime, here is the HDF header for the bin file that I've been using for my testing.  I don't know if it will be of any use, but it may be of interest until I get that file to you.   Barry
attachment 1

blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Thu Jul 16, 2020 3:51 pm America/New_York

Hi Don.  The IT folks seem to be stymied about finding a way for me to post a file for you to grab that doesn't involve you applying for an account on one of our Sharepoint sites.  To complicate things, our Sharepoint system is in the process of being migrated and isn't very reliable right now.  They said that they're still thinking about it, but in the interest of time, I was wondering if I could just e-mail a bin file to you.   They aren't really big (~500KB) and I could compress it which should squeeze it a bit if that is a problem.  If that is OK, just let me know where to send it.  If not, I'll stand by and hope they come up with something.

Thanks, Barry

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 214
Joined: Thu Mar 05, 2009 10:25 am America/New_York

l3mapgen/smigen redux

by OB SeaDAS - dshea » Thu Jul 16, 2020 7:09 pm America/New_York

Sure,

Not sure if it will work, but send it to my email:

don.shea@nasa.gov

don

blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Thu Jul 16, 2020 8:14 pm America/New_York

Thanks so much, Don.  I sent the file from my work e-mail (barry.lesht@gdit.com).   Barry

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 214
Joined: Thu Mar 05, 2009 10:25 am America/New_York

l3mapgen/smigen redux

by OB SeaDAS - dshea » Fri Jul 24, 2020 4:34 pm America/New_York

Barry,

I processed the bin file you sent me with smigen and l3mapgen.  I get the same results as you with l3mapgen having an extra line.  It looks like smigen is using a float for the resolution calculations and l3mapgen is using doubles.  The only thing I came up with that could help you is fudging the north and south lat values a little.  Sorry, not much help.

don

blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Fri Jul 24, 2020 4:38 pm America/New_York

Thanks, Don.  I'll think about the best path forward for my work.  Fudging the geographic limits might be something to try; I certainly can experiment with it.

Barry

blesht
Posts: 83
Joined: Mon Sep 19, 2005 3:06 pm America/New_York

l3mapgen/smigen redux

by blesht » Sun Jul 26, 2020 6:00 pm America/New_York

Hi Don - I've been looking at the source codes for l3mapgen and smigen.  I found the calculation in the l3mapgen routine that defines the number of rows in the output grid.  For the input file I sent you, the one that smigen yields a grid with 225 lines and l3mapgen yields one with 226 line, the l3mapgen yields rint(225.6) which apparently is rounded up (not truncated) to 226.  I have not been able to find anything equivalent to that calculation in the smigen code - the are variables with names like out_dims and Nrows, etc. but I do not see where they are calculated.  I completely understand that smigen isn't supported anymore and that you might not be familiar with the code, but if you do know where, given the input latitude limits and the desired output resolution, the number of lines in the output grid is calculated, I sure appreciate being pointed to it.  If it would be too much to ask to go through the source, that's fine.  I know it's a big ask and I'm sure you have a lot to do.  I can keep up my search.

Thanks, Barry

Post Reply