Earthdata Forum

Java performance is sensitive to memory allocations.  Both the GUI and gpt.sh use the same Graph Processing Framework (GPF) libraries, so "Proper Parameters Prevent PXXX Poor Performance".  Java memory allocation is a tricky topic, but for starters, you should compare the values of the "-Xmx" option used by the SeaDAS 7 GUI from seadas.vmoptions with the value in your .  On a system used for small teaching examples:

% grep Xmx seadas.vmoptions gpt.sh
seadas.vmoptions:# -Xmx512m
seadas.vmoptions:-Xmx1024M
gpt.sh:    -Xmx1024M \


Here, the hash (#) indicates a line that has been commented out.   These can be considered starting values.   If you encounter performance problems, try doubling the values until performance is acceptable without exceeding 60% of the system memory.   If your system has minimal RAM (4GB for linux) you may need to add RAM.   Reducing the size of your level-2 files will lower memory demands for GPF processing.  

If your gpt.sh script already has the same setting as your GUI, you may be attempting to use more memory than is installed on the system.  Linux can use swapping to temporarily stores areas of memory to disk, but this is very slow.   On lower spec machines you may not be able to use the GUI and gpt.sh efficiently at the same time.   There are many tools that can be used to monitor system memory usage.   You can use the free command in a terminal window:
$ free -m
              total        used        free      shared  buff/cache   available
Mem:           6256          63        3344           0        2849        5906
Swap:          2048           0        2048

Consult the references for your distribution for GUI tools.

In addition to possible memory allocation differences between the GUI and command line gpt run of Reprojection, the GUI run of Reprojection can be extremely fast if you choose to NOT save the target file.  In this GUI case, the target file is "virtual" and reprojection does not actually occur until a band of the target is actually viewed in the GUI, and even then reprojection only occurs in the target band being viewed, not the full file.

When running gpt, you might also choose to close the SeaDAS GUI application during the run for possible better use of the memory so as to not have 2 simultaneous instances of seadas going.

Danny

Thanks Danny.
I am running the GUI and gpt on two different computers.
I am note sure that only rasters reprojected are the ones you view! I open an L2 file (.nc) in GUI created by l2gen on my Linux machine via a batch job, just select the file and choose raster-->reproject! There is an option to save the reprojected file that I use and I do get a 'reprojected' file with all the rasters!

Thanks.
Amita

Are both computers processing files on local disks or is one using files shared on a network drive?   Does one system have a rotating magentic media disk and the other a modern solid-state disk?   Perhaps the computer you use for gpt is just older and slower than the one you use for the GUI.  Even with identical hardware, newer Java versions may be faster at some operations.   Also, Java on Windows may use runtime libraries better matched to the hardware.  On linux, Java may be built to run on a wider range of hardware and miss optimizations for recent processor models.

The L2 file format is the same for extracts (subset) as for non-extracted data, so to the reprojecton tool they are the same.  There is no issue reprojecting  data subsetted at the L2 step.

Sean

Great! Thanks for clarifying this, Sean.
Amita

This is correct Amita.  If you save the file then the reprojection occurs for all bands.  As I mentioned, if you create a "virtual" file (not saved but just in the GUI) then the reprojection does not occur until you view the band.

thanks
Danny

I was trying to reproject L2 SST files downloaded from Oceacolor website. It is showing the following error. It will be great if any of you could help.
OS: Windows
Command line code:FOR %X in (I:\DINEOF\BOB\test\*.nc) DO (gpt reproject.xml -Ssource="%X" -f 'GeoTIFF' -Ptarget="I:\DINEOF\BOB\test\test2\%~nX.tif")
Error :
I:\DINEOF\BOB\test>(gpt reproject.xml -Ssource="I:\DINEOF\BOB\test\AQUA_MODIS.20
071229T071507.L2.SST.x.nc" -f 'GeoTIFF' -Ptarget="I:\DINEOF\BOB\test\test2\AQUA_
MODIS.20071229T071507.L2.SST.x.tif" )
04:11:29.614 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:29.678 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:29.712 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:29.739 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:29.759 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:29.781 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp

Error: java.lang.NullPointerException

I:\DINEOF\BOB\test>(gpt reproject.xml -Ssource="I:\DINEOF\BOB\test\AQUA_MODIS.20
071230T080006.L2.SST.x.nc" -f 'GeoTIFF' -Ptarget="I:\DINEOF\BOB\test\test2\AQUA_
MODIS.20071230T080006.L2.SST.x.tif" )
04:11:31.687 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:31.752 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:31.787 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:31.814 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp
04:11:31.834 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp

Error: java.lang.NullPointerException

XML file used :

<graph id="ReprojectionGraph">
  <version>1.0</version>
  <node id="reprojectNode">
    <operator>Reproject</operator>
    <sources>
    <source>${source}</source>
   </sources>
    <parameters>
    <wktFile/>
    <crs>GEOGCS["WGS 84",
      DATUM["World Geodetic System 1984",
      SPHEROID["WGS 84", 6378137.0, 298.257223563, AUTHORITY["EPSG","7030"]],
      AUTHORITY["EPSG","6326"]],
      PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]],
      UNIT["degree", 0.017453292519943295],
      AXIS["Geodetic longitude", EAST],
      AXIS["Geodetic latitude", NORTH],
      AUTHORITY["EPSG","4326"]]</crs>
    <resampling>Nearest</resampling>
    <orthorectify>false</orthorectify>
    <noDataValue>NaN</noDataValue>
    <includeTiePointGrids>true</includeTiePointGrids>
    <addDeltaBands>false</addDeltaBands>
  </parameters>
  </node>
</graph>

Thanking you in advance

I downloaded the file (https://oceandata.sci.gsfc.nasa.gov/ob/getfile/AQUA_MODIS.20071229T071507.L2.SST.nc) and ran it on my mac with no problem.

I used your graph file and the command line I used is:
gpt reproject.xml -Ssource=AQUA_MODIS.20071229T071507.L2.SST.nc -f GeoTIFF -Ptarget=AQUA_MODIS.20071229T071507.L2.SST_reprojected.tif

I do see an "x" in your file name likely indicating an extract.  I don't believe this is the issue but to rule it out perhaps try running it on the unextracted file.

thanks
Danny

Hi Danny,
Thank you for your response. Unfortunately, it didn't work for me. I downloaded the file from your link and tried with your command line with the same graph file I posted earlier. However I got the following error.

I:\DINEOF\BOB\test>gpt reproject.xml -Ssource=AQUA_MODIS.20071229T071507.L2.SST.
nc -f GeoTIFF -Ptarget=AQUA_MODIS.20071229T071507.L2.SST_reprojected.tif
22:48:15.019 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5
iosp

Error: null