l2gen products

[avmehta]

Hello,
Thanks for the detail information. I use GUI on my Mac and trying run this command-line version on my Linux server which has Java version 1.7.
I installed jre1.8 on my server and changed the path in gpt.sh ($app_java_home/bin/java). Now it does have access to v1.8 and the program ran but it looks like there is some mis-match and now I get the following (I think it is all Java related?)! I do not understand the recommended fix (which libfile?).
Sorry for all the trouble.
Amita

Java HotSpot(TM) Server VM warning: You have loaded library /home/avmehta/seadas-7.5.3/modules/lib-hdf-2.7/lib/libjhdf5.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
java.lang.UnsatisfiedLinkError: /home/avmehta/seadas-7.5.3/modules/lib-hdf-2.7/lib/libjhdf5.so: /home/avmehta/seadas-7.5.3/modules/lib-hdf-2.7/lib/libjhdf5.so: wrong ELF class: ELFCLASS64 (Possible cause: architecture word width mismatch)
  at java.lang.ClassLoader$NativeLibrary.load(Native Method)
  at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
  at java.lang.Runtime.load0(Runtime.java:809)
  at java.lang.System.load(System.java:1086)
  at ncsa.hdf.hdf5lib.H5.loadH5Lib(H5.java:312)
  at ncsa.hdf.hdf5lib.H5.<clinit>(H5.java:266)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:348)
  at org.esa.beam.dataio.ProbaVProductReaderPlugIn.loadClassWithNativeDependencies(ProbaVProductReaderPlugIn.java:169)
  at org.esa.beam.dataio.ProbaVProductReaderPlugIn.loadHdf5Lib(ProbaVProductReaderPlugIn.java:157)
  at org.esa.beam.dataio.ProbaVProductReaderPlugIn.<clinit>(ProbaVProductReaderPlugIn.java:59)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  at java.lang.Class.newInstance(Class.java:442)
  at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
  at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
  at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
  at org.esa.beam.BeamCoreActivator.loadServices(BeamCoreActivator.java:62)
  at org.esa.beam.framework.dataio.ProductIOPlugInManager.<init>(ProductIOPlugInManager.java:195)
  at org.esa.beam.framework.dataio.ProductIOPlugInManager$Holder.<clinit>(ProductIOPlugInManager.java:202)
  at org.esa.beam.framework.dataio.ProductIOPlugInManager.getInstance(ProductIOPlugInManager.java:46)
  at org.esa.beam.framework.gpf.main.CommandLineArgs.detectWriterFormat(CommandLineArgs.java:313)
  at org.esa.beam.framework.gpf.main.CommandLineArgs.parseArgs(CommandLineArgs.java:186)
  at org.esa.beam.framework.gpf.main.CommandLineArgs.parseArgs(CommandLineArgs.java:69)
  at org.esa.beam.framework.gpf.main.CommandLineTool.run(CommandLineTool.java:117)
  at org.esa.beam.framework.gpf.main.GPT.run(GPT.java:54)
  at org.esa.beam.framework.gpf.main.GPT.main(GPT.java:34)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at com.bc.ceres.launcher.Launcher.launch(Launcher.java:154)
  at com.bc.ceres.launcher.Launcher.main(Launcher.java:56)
java.lang.UnsatisfiedLinkError: no jhdf5 in java.library.path
  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
  at java.lang.Runtime.loadLibrary0(Runtime.java:870)
  at java.lang.System.loadLibrary(System.java:1122)
  at ncsa.hdf.hdf5lib.H5.loadH5Lib(H5.java:339)
  at ncsa.hdf.hdf5lib.H5.<clinit>(H5.java:266)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:348)
  at org.esa.beam.dataio.ProbaVProductReaderPlugIn.loadClassWithNativeDependencies(ProbaVProductReaderPlugIn.java:169)
  at org.esa.beam.dataio.ProbaVProductReaderPlugIn.loadHdf5Lib(ProbaVProductReaderPlugIn.java:157)
  at org.esa.beam.dataio.ProbaVProductReaderPlugIn.<clinit>(ProbaVProductReaderPlugIn.java:59)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  at java.lang.Class.newInstance(Class.java:442)
  at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
  at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
  at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
  at org.esa.beam.BeamCoreActivator.loadServices(BeamCoreActivator.java:62)
  at org.esa.beam.framework.dataio.ProductIOPlugInManager.<init>(ProductIOPlugInManager.java:195)
  at org.esa.beam.framework.dataio.ProductIOPlugInManager$Holder.<clinit>(ProductIOPlugInManager.java:202)
  at org.esa.beam.framework.dataio.ProductIOPlugInManager.getInstance(ProductIOPlugInManager.java:46)
  at org.esa.beam.framework.gpf.main.CommandLineArgs.detectWriterFormat(CommandLineArgs.java:313)
  at org.esa.beam.framework.gpf.main.CommandLineArgs.parseArgs(CommandLineArgs.java:186)
  at org.esa.beam.framework.gpf.main.CommandLineArgs.parseArgs(CommandLineArgs.java:69)
  at org.esa.beam.framework.gpf.main.CommandLineTool.run(CommandLineTool.java:117)
  at org.esa.beam.framework.gpf.main.GPT.run(GPT.java:54)
  at org.esa.beam.framework.gpf.main.GPT.main(GPT.java:34)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at com.bc.ceres.launcher.Launcher.launch(Launcher.java:154)
  at com.bc.ceres.launcher.Launcher.main(Launcher.java:56)
org.esa.beam.framework.gpf.OperatorException: Cannot construct DataBuffer.
  at org.esa.beam.framework.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:367)
  at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
  at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
  at javax.media.jai.OpImage.getTile(OpImage.java:1129)
  at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
  at javax.media.jai.RenderedOp.getData(RenderedOp.java:2276)
  at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
  at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:378)
  at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:364)
  at org.esa.beam.framework.gpf.internal.OperatorImage.computeRect(OperatorImage.java:70)
  at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
  at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
  at javax.media.jai.OpImage.getTile(OpImage.java:1129)
  at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
  at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: java.lang.RuntimeException: Cannot construct DataBuffer.
  at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(DataBufferUtils.java:132)
  at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(DataBufferUtils.java:214)
  at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(ComponentSampleModelJAI.java:271)
  at javax.media.jai.RasterFactory.createWritableRaster(RasterFactory.java:691)
  at javax.media.jai.PlanarImage.createWritableRaster(PlanarImage.java:1982)
  at javax.media.jai.PlanarImage.getExtendedData(PlanarImage.java:2456)
  at com.sun.media.jai.opimage.WarpBilinearOpImage.computeRectFloat(WarpBilinearOpImage.java:511)
  at com.sun.media.jai.opimage.WarpBilinearOpImage.computeRect(WarpBilinearOpImage.java:112)
  at javax.media.jai.WarpOpImage.computeTile(WarpOpImage.java:577)
  at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
  ... 12 more

Error: Cannot construct DataBuffer.

[gnwiii]

Glad you were able to make some progress.

The error: java.lang.UnsatisfiedLinkError: [...]: wrong ELF class: ELFCLASS64 (Possible cause: architecture word width mismatch) might mean you have installed SeaDAS (only available as 64-bit) with 32-bit linux and/or Java Runtime Environment (JRE) software.   More details of your linux and Java installation might help.

On a 64-bit linux with 64-bit seadas, and 64-bit JRE I get:

% uname -a
Linux cerberus 5.5.17-200.fc31.x86_64 #1 SMP Mon Apr 13 15:29:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
indicating 64-bit linux (my 32-bit system has i686 in place of x86_64, you might see amd64).

% file seadas-7*/modules/lib-hdf-2.7/lib/libjhdf5.so
seadas-7*/modules/lib-hdf-2.7/lib/libjhdf5.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped,
which indicates a 64-bit installation of SeaDAS 7, and (using the JRE from ESA SNAP -- you will probably use something like /usr/bin/java):

% ~/snap/jre/bin/java -version
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode),
which indicates a 64-bit JRE.

[avmehta]

Hi,
  The Linux server I use a 64-bit machine. I installed java for 64-bit (previously I  installed was for 32-bit).  Here is what I get for my java version:

java -version
java version "1.8.0_251"
Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)

Now when I run gpt.sh I get the following (the wrong ELF class error is gone) but the DataBuffer error it still there.

org.esa.beam.framework.gpf.OperatorException: Cannot construct DataBuffer.
  at org.esa.beam.framework.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:367)
  at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
  at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
  at javax.media.jai.OpImage.getTile(OpImage.java:1129)
  at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
  at javax.media.jai.RenderedOp.getData(RenderedOp.java:2276)
  at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
  at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:378)
  at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:364)
  at org.esa.beam.framework.gpf.internal.OperatorImage.computeRect(OperatorImage.java:70)
  at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
  at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
  at javax.media.jai.OpImage.getTile(OpImage.java:1129)
  at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
  at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: java.lang.RuntimeException: Cannot construct DataBuffer.
  at com.sun.media.jai.util.DataBufferUtils.constructDataBuffer(DataBufferUtils.java:132)
  at com.sun.media.jai.util.DataBufferUtils.createDataBufferFloat(DataBufferUtils.java:214)
  at javax.media.jai.ComponentSampleModelJAI.createDataBuffer(ComponentSampleModelJAI.java:271)
  at javax.media.jai.RasterFactory.createWritableRaster(RasterFactory.java:691)
  at javax.media.jai.PlanarImage.createWritableRaster(PlanarImage.java:1982)
  at javax.media.jai.PlanarImage.getExtendedData(PlanarImage.java:2456)
  at com.sun.media.jai.opimage.WarpBilinearOpImage.computeRectFloat(WarpBilinearOpImage.java:511)
  at com.sun.media.jai.opimage.WarpBilinearOpImage.computeRect(WarpBilinearOpImage.java:112)
  at javax.media.jai.WarpOpImage.computeTile(WarpOpImage.java:577)
  at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
  ... 12 more

So the Linux machine I have still has Java 1.7 and do not have root password t uninstall and install the new version.  Can this cause problem?
Thanks so much for your help.
Amita

[gnwiii]

The org.esa.beam.framework.gpf.OperatorException: Cannot construct DataBuffer. error means Java needs more memory.  The gpt.sh script sets a 1G limit, which is fairly small, but if your virtual machine has less than 4GB of memory it may not be able to run GPT processing.

[avmehta]

Oh - I remember now that I had similar  issue on my GUI when reprojecting and had to change the memory. I have increased the memory in both seadas.vmoptions and in gpt.sh -- the program now seems to be running!
Thank you so very much.
Amita

[avmehta]

Hello,
  I was able to run gpt.sh by increasing java memory! Thanks.
  For reprojecting OLI L2 file I used the following par file --  same as given in  the cookbook, but changed the resolution to be 30m (in degree).

     pixelSize=0.000274658
      resampling=Bilinear
     maskExpression=!l2_flags.HIGLINT and !l2_flags.HISATZEN and !l2_flags.LOWLW and !l2_flags.HILT
      transferValidPixelExpression=true
      applyValidPixelExpression=true

  Can you please  point to a documentation about the par file? I am not sure what 'maskExpression' does!
  I am confused as the reprojection is taking way too long on my Linux server compared to doing it through SeaDAS GUI on my Mac (is almost 3 to 4 times longer)! Is that usually the case?
   Also, the resulting reprojected files from the GUI and from gpt are different -- which is I think is because the GUI has default of 'nearest neighbor' sampling whereas the par file I used has 'bilinear sampling'.
   Now I am working on latlon2pixline to select a smaller domain for L1 to L2 conversion. I believe still the output will have the entire image tile - just the box I provide will be converted to L2. Is there a way to tell gpt.sh to reproject to work on only the lat-lat on box?
  Thanks.
  Amita

[gnwiii]

The .par file is just a convenient way to without a very long command line or editing the XML graph.  The parameters specified in the .par file depend on the operator.  Run gpt<.ext> <operator> -h to see a list of parameters.  For "Reproject", expressions follow the same rules you would use for "Band Maths" or "Valid pixel" in the SeaDAS GUI.  You should be able to see the effects by using a similar expression to define "Valid Pixels" in the GUI or by playing with flag selections.  It looks like the names in the graph and the GPT help are not fully consistent.   On linux:

% ~/seadas-7/bin/gpt.sh Reproject -h | grep Expression
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

  -PapplyValidPixelExpression=<boolean>       Apply the valid pixel expression to determine whether a source pixel gets used
  -PmaskExpression=<string>                   Logical expression of masks and/or bands to apply to determine whether a source pixel gets used
  -PtransferValidPixelExpression=<boolean>    Copy source valid pixel expression(s) to the corresponding bands of the reprojected file
        <applyValidPixelExpression>boolean</applyValidPixelExpression>
        <transferValidPixelExpression>boolean</transferValidPixelExpression>
        <maskExpression>string</maskExpression>

You can see that the command line "-PmaskExpression=<string>" matches the XML snippet: "<maskExpression>string</maskExpression>".

[OB.DAAC - SeanBailey]

> Now I am working on latlon2pixline to select a smaller domain for L1 to L2 conversion. I believe still the output will have the entire image tile - just the box I provide will be converted to L2. Is there a way to tell gpt.sh to reproject to work on only the lat-lat on box?

If you apply the spixel/epixel sline/eline parameters output from lonlat2pixline to the L2 processing step, the resulting L2 file will only include this region of interest.

Sean

[avmehta]

Thanks a lot for the explanation. I am still wondering why the gpt reprojection would take such a long time compared to the GUI processing! Do you know if that is a common experience? May be GUI uses different defaults ?
Thanks again.
Amita

[avmehta]

Hello,
Thanks for the information.  I would  run l2gen with these options.
  I just want to make sure that I can then repoeject the cropped L2 data. I seem to remember from some comment in the forum that entire image tile is required  for reprojection.
Amita