Welcome to the Earthdata Forum! Here, the scientific user community and subject matter experts from NASA Distributed Active Archive Centers (DAACs), and other contributors, discuss research needs, data, and data applications.
by gnwiii » Mon Apr 27, 2020 1:09 pm America/New_York
Java performance is sensitive to memory allocations. Both the GUI and gpt.sh use the same Graph Processing Framework (GPF) libraries, so "Proper Parameters Prevent PXXX Poor Performance". Java memory allocation is a tricky topic, but for starters, you should compare the values of the "-Xmx" option used by the SeaDAS 7 GUI from seadas.vmoptions with the value in your . On a system used for small teaching examples:
Here, the hash (#) indicates a line that has been commented out. These can be considered starting values. If you encounter performance problems, try doubling the values until performance is acceptable without exceeding 60% of the system memory. If your system has minimal RAM (4GB for linux) you may need to add RAM. Reducing the size of your level-2 files will lower memory demands for GPF processing.
If your gpt.sh script already has the same setting as your GUI, you may be attempting to use more memory than is installed on the system. Linux can use swapping to temporarily stores areas of memory to disk, but this is very slow. On lower spec machines you may not be able to use the GUI and gpt.sh efficiently at the same time. There are many tools that can be used to monitor system memory usage. You can use the free command in a terminal window: $ free -m total used free shared buff/cache available Mem: 6256 63 3344 0 2849 5906 Swap: 2048 0 2048
Consult the references for your distribution for GUI tools.
In addition to possible memory allocation differences between the GUI and command line gpt run of Reprojection, the GUI run of Reprojection can be extremely fast if you choose to NOT save the target file. In this GUI case, the target file is "virtual" and reprojection does not actually occur until a band of the target is actually viewed in the GUI, and even then reprojection only occurs in the target band being viewed, not the full file.
When running gpt, you might also choose to close the SeaDAS GUI application during the run for possible better use of the memory so as to not have 2 simultaneous instances of seadas going.
by avmehta » Mon Apr 27, 2020 4:04 pm America/New_York
Thanks Danny. I am running the GUI and gpt on two different computers. I am note sure that only rasters reprojected are the ones you view! I open an L2 file (.nc) in GUI created by l2gen on my Linux machine via a batch job, just select the file and choose raster-->reproject! There is an option to save the reprojected file that I use and I do get a 'reprojected' file with all the rasters!
by gnwiii » Mon Apr 27, 2020 5:20 pm America/New_York
Are both computers processing files on local disks or is one using files shared on a network drive? Does one system have a rotating magentic media disk and the other a modern solid-state disk? Perhaps the computer you use for gpt is just older and slower than the one you use for the GUI. Even with identical hardware, newer Java versions may be faster at some operations. Also, Java on Windows may use runtime libraries better matched to the hardware. On linux, Java may be built to run on a wider range of hardware and miss optimizations for recent processor models.
This is correct Amita. If you save the file then the reprojection occurs for all bands. As I mentioned, if you create a "virtual" file (not saved but just in the GUI) then the reprojection does not occur until you view the band.
by swathy » Wed May 20, 2020 6:57 pm America/New_York
I was trying to reproject L2 SST files downloaded from Oceacolor website. It is showing the following error. It will be great if any of you could help. OS: Windows Command line code:FOR %X in (I:\DINEOF\BOB\test\*.nc) DO (gpt reproject.xml -Ssource="%X" -f 'GeoTIFF' -Ptarget="I:\DINEOF\BOB\test\test2\%~nX.tif") Error : I:\DINEOF\BOB\test>(gpt reproject.xml -Ssource="I:\DINEOF\BOB\test\AQUA_MODIS.20 071229T071507.L2.SST.x.nc" -f 'GeoTIFF' -Ptarget="I:\DINEOF\BOB\test\test2\AQUA_ MODIS.20071229T071507.L2.SST.x.tif" ) 04:11:29.614 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5 iosp 04:11:29.678 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5 iosp 04:11:29.712 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5 iosp 04:11:29.739 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5 iosp 04:11:29.759 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5 iosp 04:11:29.781 [main] DEBUG ucar.nc2.NetcdfFile - Using IOSP ucar.nc2.iosp.hdf5.H5 iosp
by swathy » Fri May 22, 2020 1:17 pm America/New_York
Hi Danny, Thank you for your response. Unfortunately, it didn't work for me. I downloaded the file from your link and tried with your command line with the same graph file I posted earlier. However I got the following error.