Page 1 of 3

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 9:46 am America/New_York
by bruce
Notes on building on 18.04

Only snag so far is that xdr won't build as shipped on 18.  It could probably be fixed with an additional option on the command line to compile it, but the "real" fix is to edit the failing test routine (opt/src/xdr/bsd-xdr-1.0.0/src/test/xdrstdio_test.c) to remove 2 spaces from the in front of the "return" at line 243.

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 9:47 am America/New_York
by bruce
Stuck now at sdptk,  BuildIt fails with

***********************************
********* Building SDPTK **********
***********************************
patching file bin/INSTALL
patching file bin/INSTALL-Toolkit
patching file include/libxml/Makefile
patching file include/libxml/xmlversion.h
patching file include/PGS_SMF.f
patching file include/PGS_SMF.h
patching file src/EPH/orbsim/makefile
patching file src/XML/xml/Makefile

Toolkit Installation starting at Tue Jun 19 09:29:18 EDT 2018

Note: If your system is CentOS linux, you may need -fPIC compilation flag.
If that is the case you better quit here (using Ctrl-c) and re-start
installation with additional arguments -cc_flags "-fPIC" and/or
-cpp_flags "-fPIC" and -fc_flags "-fPIC" if you have not done so.
You may type bin/INSTALL -h to see notes on how to add additional
compilation flags
The default Toolkit LINUX installation for this machine is 64 bit. Other
available Toolkit LINUX installation mode is 32 bit Mode

Installing in  (64 bit) mode...

./makefile.cpp
...file modified
include/makefile.cpp
...file modified
include/CUC/makefile.cpp
...file modified
include/DCW/makefile.cpp
...file modified
include/FF/makefile.cpp
...file modified
include/FW/makefile.cpp
...file modified
src/makefile.cpp
...file modified
src/AA/makefile.cpp
...file modified
src/AA/DCW/makefile.cpp
...file modified
src/AA/VPF/makefile.cpp
...file modified
src/AA/freeform/makefile.cpp
...file modified
src/AA/generic/makefile.cpp
...file modified
src/CBP/makefile.cpp
...file modified
src/CSC/makefile.cpp
...file modified
src/CUC/makefile.cpp
...file modified
src/CUC/ODL/makefile.cpp
...file modified
src/CUC/UDUNITS/makefile.cpp
...file modified
src/EPH/makefile.cpp
...file modified
src/EPH/orbsim/makefile.cpp
...file modified
src/EPH/gbadsim/makefile.cpp
...file modified
src/GCT/makefile.cpp
...file modified
src/GCT/freeware/makefile.cpp
...file modified
src/IO/makefile.cpp
...file modified
src/IO/GEN/makefile.cpp
...file modified
src/IO/L0/makefile.cpp
...file modified
src/IO/L0/L0sim/makefile.cpp
...file modified
src/MEM/makefile.cpp
...file modified
src/MET/makefile.cpp
...file modified
src/MET/support/makefile.cpp
...file modified
src/MET/tools/makefile.cpp
...file modified
src/PC/makefile.cpp
...file modified
src/SMF/makefile.cpp
...file modified
src/TD/makefile.cpp
...file modified
src/DEM/makefile.cpp
...file modified
src/TSF/makefile.cpp
...file modified
src/XML/makefile.cpp
...file modified
src/XML/xml/Makefile.cpp
...file modified
src/XML/xsl/Makefile.cpp
...file modified
message/makefile.cpp
...file modified

Setting ZLIB home directory to: /seadas/lcl/ocssw/opt

Setting JPEG home directory to: /seadas/lcl/ocssw/opt

Setting SZIP home directory to: /seadas/lcl/ocssw/opt
Setting LD_LIBRARY_PATH to /seadas/lcl/ocssw/opt/lib

Setting HDF home directory to: /seadas/lcl/ocssw/opt

***** Copying JPEG, ZLIB and SZIP header and library *********

********** files to relevant directories in HDF.    **********

Setting HDF5 home directory to: /seadas/lcl/ocssw/opt
Adding /seadas/lcl/ocssw/opt/lib to LD_LIBRARY_PATH

Setting HDFEOS home directory to: /seadas/lcl/ocssw/opt/EOS

Setting HDFEOS5 home directory to: /seadas/lcl/ocssw/opt/EOS

Running the Toolkit Installation Script ...

Running the Toolkit Installation Script without AA Tool.
bin/INSTALL: 3190: bin/INSTALL: /seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit: not found
bin/INSTALL: 3192: bin/INSTALL: cannot create /seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin/linux64/pgs-env.csh: Directory nonexistent

INSTALL: Error: The Toolkit Installation Script failed or had errors.
SDP Toolkit installation cancelled
***** INSTALL failed *****

What is the AA tool?  Is that a part of ubuntu that I need to install or something else?

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 10:10 am America/New_York
by bruce
but the first file that's "not found" appears to be there...

bcb@ubuntu:/seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin$ ls
common        INSTALL-HDF4.0r2  INSTALL-HDF4.2r10  INSTALL-HDF4.X      INSTALL-HDF5-1.6.2   INSTALL-HDF5-1.8.12  INSTALL-JPEGSRC.v6b  INSTALL-ZLIB   pgs-flags
ftp.csh        INSTALL-HDF4.1r1  INSTALL-HDF4.2r2   INSTALL-HDF5       INSTALL-HDF5-1.6.3   INSTALL-HDF5-1.8.3  INSTALL-SZIP       INSTALL-ZLIB-1.2.1  pgs-flags-cpp
INSTALL        INSTALL-HDF4.1r3  INSTALL-HDF4.2r3   INSTALL-HDF5-1.4.0  INSTALL-HDF5-1.6.4   INSTALL-HDF5-1.8.5  INSTALL-SZIP1.1      INSTALL-ZLIB-1.2.3  tmp
INSTALL-4.1r3_aix   INSTALL-HDF4.1r4  INSTALL-HDF4.2r4   INSTALL-HDF5-1.4.1  INSTALL-HDF5-1.6.5   INSTALL-HDF5-1.8.8  INSTALL-SZIP1.2      INSTALL-ZLIB-1.2.5
INSTALL-AAdata      INSTALL-HDF4.1r5  INSTALL-HDF4.2r5   INSTALL-HDF5-1.4.3  INSTALL-HDF5-1.6.6   INSTALL-HDFEOS5-Wrap  INSTALL-SZIP2.0      INSTALL-ZLIB-1.2.8
INSTALL-HDF      INSTALL-HDF4.2r0  INSTALL-HDF4.2r6   INSTALL-HDF5-1.4.4  INSTALL-HDF5-1.6.7   INSTALL-HDFEOS-Wrap  INSTALL-SZIP2.1      mkpgslib
INSTALL-HDF4.0r1p1  INSTALL-HDF4.2r1  INSTALL-HDF4.2r9   INSTALL-HDF5-1.6.1  INSTALL-HDF5-1.8.11  INSTALL-JPEG    INSTALL-Toolkit      mkpgslib.cpp

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 2:35 pm America/New_York
by gfireman
The AA tool is a part of the Science Data Toolkit that we don't use, so that's not the problem.

This is really bizarre.  Is /seadas/lcl on a remote disk, that maybe just glitched out for a second?
Please try:
ls -l $LIB3_DIR/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit
Mine looks like:
-rwxr-xr-x 1 gfireman gfireman 177704 2013-12-12 11:59 /accounts/gfireman/ocssw/opt/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit

Which shell are you using?

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 3:02 pm America/New_York
by bruce
bcb@modis:~$ ls -l $LIB3_DIR/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit
-rwxr-xr-x 1 bcb bcb 177265 Jun 19 10:06 /seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit
bcb@modis:~$

bash, it's the only shell worth using (I know them's fightin words to some :-)

It's a local disk, and the failure is repeatable

Everything else that comes after it in the buildit sequence (other than mapi and sdst which appear to depend on it) build just fine.

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 3:28 pm America/New_York
by gfireman
I use bash too :)
Your file is post-patch, which accounts for the different date and size.

The ECS tools page lists a newer version of the SDPTK, but I don't know if it will work any better.  We are still running Ubuntu 16.04, so I don't have a way to test either installation.  (Don could, but he's been called away this week).

The hdfeos, hdfeos5, sdptk, mapi, and sdst libraries are used only in generating MODIS L1A, GEO, and L1B.  If you don't need those, we should be able to come up with a way to compile the rest of OCSSW without them.

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 3:44 pm America/New_York
by bruce
For the time being, I'm mostly interested in mucking around in l2gen, so as long as I can rebuild that, I should be happy (at least for a little while :-) )

Building on Ubuntu 18.04

Posted: Tue Jun 19, 2018 5:17 pm America/New_York
by gfireman
I believe the following will work.  It compiles, but I haven't tested it.

1. In file $LIB3_DIR/src/BuildIt
remove from the subdirs list:  hdfeos hdfeos5 sdptk mapi sdst

2. Delete from directory $OCSSWROOT/cmake/Modules
FindHDFEOS.cmake
FindHDFEOS5.cmake
FindMapi.cmake
FindPGSTK.cmake
FindSDST.cmake

3. In file $OCSSWROOT/src/CMakeLists.txt
delete or comment out all the "add_subdirectory" lines in the "MODIS-specific" section.

Building on Ubuntu 18.04

Posted: Wed Jun 20, 2018 9:05 am America/New_York
by bruce
presuming the next steps are still 

cd build
cmake ..
make -j 20


It runs for a while and then errors out as in the attached log file
attachment 1

Building on Ubuntu 18.04

Posted: Wed Jun 20, 2018 10:20 am America/New_York
by gfireman
After making those changes, I'd start fresh: delete the existing build directory, then follow the build instructions in README.md, omitting the download steps.
I'd also use just "make" instead of "make -j 20".   Leaving out the "-j" parameter will run only one job:  slower, but it's easier to read a linear log when there's trouble.

Turns out we do have an Ubuntu 18.04 system; I'll see if I can get the standard compile to work there.