Plans to add EPIC/DSCOVR to SeaDAS?

[jdmcpherson]

Hi, we're working with EPIC/DSCOVR data, and need to perform standard tasks on it now done by SeaDAS with other satellites, such as L3 binning, remapping to L3smi rectilinear global grid, etc.
Is there a way to read and manipulate this data directly with SeaDAS now?
If not, is there a plan to add the satellite soon?
If not, is there a way to adapt the relevant SeaDAS routines (and which ones?) to at least put the EPIC data on an equal-area grid (e.g., L3bin sinusoidal), and then to remap it to the standard L3smi global grid?
If not, is there other software available somewhere  to accurately perform these tasks?
If not, I'll have to write programs to do it, so where can I find the exact specs/algorithms to generate the L3bin equal-area grid, and the algo needed to remap from that equal-area grid to the L3smi rectilinear global grid (e.g., 2160x1080, or 4320x2160, or 8640x4320 used for the L3 MODIS products)?

We need to transfer the EPIC data to some standard global grid because we're using multiple observations per day per area/pixel to accurately estimate PAR.
Previously, I wrote a program using nearest-neighbor matching to transfer the EPIC data to the L3 MODIS global grid, but my PI didn't like the results and said I need to find out how do it with much greater accuracy and less distortion/noise for the various sensors (and EPIC if you have it).

Thanks, John

[OB.DAAC - SeanBailey]

> Is there a way to read and manipulate this data directly with SeaDAS now?

Don't know, never tried :grin:  Have you?  I believe they are HDF5 files, so it's possible ...a lot depends upon how the files were structured.

>If not, is there a plan to add the satellite soon?

No plans, but that doesn't mean we can't be persuaded to do so.  Won't be anytime soon, though....way too busy at the moment...

>If not, is there a way to adapt the relevant SeaDAS routines (and which ones?) to at least put the EPIC data on an equal-area grid (e.g., L3bin sinusoidal), and then to remap it to the standard L3smi global grid?

There's always a way.  If you can get the EPIC data into our L2 file format, then l2bin could work on it.  Right now, l2bin expects to know the sensor - so you'd have to fake it.  A future version of the binner will be more agnostic.

> If not, is there other software available somewhere  to accurately perform these tasks?

No clue.

> If not, I'll have to write programs to do it, so where can I find the exact specs/algorithms to generate the L3bin equal-area grid, and the algo needed to remap from that equal-area grid to the L3smi rectilinear global grid (e.g., 2160x1080, or 4320x2160, or 8640x4320 used for the L3 MODIS products)?

https://oceancolor.gsfc.nasa.gov/docs/ocssw/ ...but I guess you figured I'd say that...

Sean

[jdmcpherson]

Thanks for the reply, Sean.

>> Is there a way to read and manipulate this data directly with SeaDAS now?
>Don't know, never tried ?  Have you?  I believe they are HDF5 files, so it's possible ...a lot depends upon how the files were structured.

Looks like trying SeaDAS on EPIC L1b fails with an error:
Type: java.util.concurrent.ExecutionException
Message: java.lang.illegalArgumentException: The product 'epic_1b_20180101001751_02' already contains a band with the name 'image'

Trying it on EPIC L2 had some measure of success:
reading 'DSCOVR_EPIC_L2_AER_01_20180101001751_02.he5'
shows tree nodes: metadata, rasters(2048x2048 in size),
and the latter shows AerosolType, CloudFraction, CloudOpticalDepth, FinalAerosolLayerHeight, Latitude, Longitude, RelativeAzimuthAngle, SolarZenithAngle, ViewingZenithAngle, etc.
viewing one shows the filled earth circle in the middle of the image, with NaN outside of the circle for "Space".
The cursor indicates:
leftmost point of the circle 237x,999y to rightmost 1813x,999y
topmost 999x,235y to bottommost 999x,1813y
Latitude is high(90) on left to low on right(-90).
Longitude has horizontal line about in middle, below it is -180 going down to -93 at bottommost point,
above the line it is 180 going up to 101 at topmost point.
So, the images are rotated 90 degrees and flipped left to right relative to what I imagine SeaDAS expects, so I'm guessing it might not work with the SeaDAS routines as they are now.

>>If not, is there a plan to add the satellite soon?
>No plans, but that doesn't mean we can't be persuaded to do so.  Won't be anytime soon, though....way too busy at the moment...

Ok, we'll ask that your team add it when you can.
In the meantime, I suspect I'll have to write some temporary programs to do what we need to do now, since our postdoc is waiting for me...

>If you can get the EPIC data into our L2 file format, then l2bin could work on it.

Can the L2/L3 routines work with global data with NaN Space values (and no lat/lon associated with Space pixels)?

>Right now, l2bin expects to know the sensor - so you'd have to fake it.

If we could get the data into a L1/L2 format that SeaDAS would accept, which sensor should we specify (that has the best chance of success with EPIC data)?

>>If not, I'll have to write programs to do it, so where can I find the exact specs/algorithms to generate the L3bin equal-area grid, and the algo needed
>>to remap from that equal-area grid to the L3smi rectilinear global grid (e.g., 2160x1080, or 4320x2160, or 8640x4320 used for the L3 MODIS products)?
> https://oceancolor.gsfc.nasa.gov/docs/ocssw/ ...but I guess you figured I'd say that...

Can you be more specific? I'm getting lost in all that.
Ideally, I'd like a file with all the L3 equal-area bins (integerized sinusoidal grid) and the association of each bin with the latitude/longitude at its center.
From there I'd put them into a 4320x2160 global grid with the northmost pixel at (2160,0) and southmost at (2160,2160) so I can conceptually see the equal-angle data projections like the top image at:
https://en.wikipedia.org/wiki/Sinusoidal_projection
To reproject this to the SMI equal-angle L3 grid, I imagine that pixels on each line will stay on that same line, but I need to know how each line gets projected to the relevant line of the SMI grid. There must be a fairly simple algorithm, but I don't know what it is. Can you point me directly to it?

I found this page, which has a perl script at the bottom (I don't know perl, so I don't quite understand the example yet) which seems to at least get the correspondence between each L3 bin and its latitude/longitude:
https://oceancolor.gsfc.nasa.gov/docs/format/l3bins/
but we still need the algo/code to go from L3 bin to L3 SMI coordinates.
We want to use the exact relation you use (with 64bit computations), to facilitate comparing EPIC to MODIS and other sensors.
So, we need to duplicate the SMI remapping you do for "4.64-kilometer Bins" (since I believe max resolution for EPIC is 20 km^2).

Thanks, John

[OB.DAAC - SeanBailey]

John,
So, seems we'll need an EPIC-specific reader for the GUI...

> Can the L2/L3 routines work with global data with NaN Space values (and no lat/lon associated with Space pixels)?

Yes.

> If we could get the data into a L1/L2 format that SeaDAS would accept, which sensor should we specify (that has the best chance of success with EPIC data)?

Doesn't matter, but you might want to avoid SeaWiFS... the binner has extra logic to deal with the SeaWiFS orbit drift...no need to use that :grin: The binner code was designed to generate "data-day" composites for polar orbiting sensors.  Geostationary is a completely different beast.  When binning, choose prodtype=regional (yeah, not intuitive in this case...)

> Can you be more specific? I'm getting lost in all that.
> Ideally, I'd like a file with all the L3 equal-area bins (integerized sinusoidal grid) and the association of each bin with the latitude/longitude at its center.

If you can mimic our L2 format, then l2bin/l3mapgen will be your friend.  This would be the easiest approach to take. 
Replicating the ISIN grid and then the code to interpret that to reproject to the SMI (plate carree/equal angular) projection would be reinventing the wheel ...but the parts to do that, should you choose to go that route, are in oel_hdf/libbin,  oel_hdf/libbin++,  src/l2bin and src/l3mapgen

Sean