Speed up SeaDAS GPT processing

Please enter here to ask a question about any NASA Science related topics!
Post Reply
jvaldezch
Posts: 26
Joined: Fri Aug 24, 2018 2:52 pm America/New_York

Speed up SeaDAS GPT processing

by jvaldezch » Wed Sep 09, 2020 10:14 pm America/New_York

Hi forum,

I'm running some gpt commands (using XML template) to do some band math over L2 at 250 meter resolution, but processing it takes too much to export data to GeoTiff, as you can see in the code below I'm also doing a reprojection when executing the commands. For example it takes about 40 min to execute

I have another gpt XML template that I have only exports and reprojects a single band, but still processing time it's very slow.

Any recommendations, perhaps using GDAL or any other external tool, or approach to solve this issue.


<graph id="fai_processing">
<version>1.0</version>
    <node id="reprojectNode">
        <operator>Reproject</operator>
        <sources>
            <source>${source}</source>
        </sources>
        <parameters>
            <crs>
               GEOGCS["WGS84(DD)",
                      DATUM["WGS84",
                            SPHEROID["WGS84", 6378137.0, 298.257223563]],
                      PRIMEM["Greenwich", 0.0],
                      UNIT["degree", 0.017453292519943295],
                      AXIS["Geodetic longitude", EAST],
                      AXIS["Geodetic latitude", NORTH]]
            </crs>
            <resampling>Nearest</resampling>
            <orientation>0.0</orientation>
            <pixelSizeX>0.002250022500225</pixelSizeX>
            <pixelSizeY>0.002250022500225</pixelSizeY>
            <orthorectify>false</orthorectify>
            <noDataValue>NaN</noDataValue>
            <includeTiePointGrids>true</includeTiePointGrids>
            <addDeltaBands>false</addDeltaBands>
            <maskExpression>!l2_flags.LAND and !l2_flags.HILT and !l2_flags.CLDICE and !l2_flags.STRAYLIGHT and !l2_flags.CHLFAIL</maskExpression>
            <transferValidPixelExpression>true</transferValidPixelExpression>
            <applyValidPixelExpression>true</applyValidPixelExpression>
        </parameters>
    </node>
    <node id="bandMathNode">
        <operator>BandMaths</operator>
        <sources>
            <source>reprojectNode</source>
        </sources>
        <parameters>
            <targetBands>
                <targetBand>
                    <name>fai</name>
                    <expression>rhos_859 - (rhos_645 + (rhos_1240 - rhos_645) * ((859 - 645) / (1240 - 645)))</expression>
                    <description>Floating Algae Index</description>
                    <type>float32</type>
                    <noDataValue>-32767.0</noDataValue>
                </targetBand>
            </targetBands>
        </parameters>
    </node>
</graph>


Thanks again!

Tags:

gnwiii
Posts: 641
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

Speed up SeaDAS GPT processing

by gnwiii » Thu Sep 10, 2020 7:11 am America/New_York

If you are using the default value for "-Xmx" in gpt.sh you may get big improvements in performance by increasing this value.  Lowering the resolution will save a lot of processing time.   You are remapping and then doing band maths.   Remapping can generate values that are never observed, so in general it is better to do bandmaths on the raw level-2 data.   For MODIS, only a few bands have 250m resolution; the others are interpolated, so there is little extra information in 250m data and some values may be unreal due to the interpolation.   It isn't hard to add a product to l2gen -- the steps are documented in the source code. You can also use python or R to add a product to an existing level-2 file.   If you add the new product to the products.xml file you can take advantage of the OCSSW standard workflows for binning and mapping.

jvaldezch
Posts: 26
Joined: Fri Aug 24, 2018 2:52 pm America/New_York

Speed up SeaDAS GPT processing

by jvaldezch » Thu Sep 10, 2020 6:53 pm America/New_York

Thanks I will try this. :smile:

Post Reply