Building on Ubuntu 18.04

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
bruce
Posts: 85
Joined: Thu Mar 17, 2005 4:36 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by bruce » Tue Jun 19, 2018 9:46 am America/New_York

Notes on building on 18.04

Only snag so far is that xdr won't build as shipped on 18.  It could probably be fixed with an additional option on the command line to compile it, but the "real" fix is to edit the failing test routine (opt/src/xdr/bsd-xdr-1.0.0/src/test/xdrstdio_test.c) to remove 2 spaces from the in front of the "return" at line 243.

Tags:

bruce
Posts: 85
Joined: Thu Mar 17, 2005 4:36 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by bruce » Tue Jun 19, 2018 9:47 am America/New_York

Stuck now at sdptk,  BuildIt fails with

***********************************
********* Building SDPTK **********
***********************************
patching file bin/INSTALL
patching file bin/INSTALL-Toolkit
patching file include/libxml/Makefile
patching file include/libxml/xmlversion.h
patching file include/PGS_SMF.f
patching file include/PGS_SMF.h
patching file src/EPH/orbsim/makefile
patching file src/XML/xml/Makefile

Toolkit Installation starting at Tue Jun 19 09:29:18 EDT 2018

Note: If your system is CentOS linux, you may need -fPIC compilation flag.
If that is the case you better quit here (using Ctrl-c) and re-start
installation with additional arguments -cc_flags "-fPIC" and/or
-cpp_flags "-fPIC" and -fc_flags "-fPIC" if you have not done so.
You may type bin/INSTALL -h to see notes on how to add additional
compilation flags
The default Toolkit LINUX installation for this machine is 64 bit. Other
available Toolkit LINUX installation mode is 32 bit Mode

Installing in  (64 bit) mode...

./makefile.cpp
...file modified
include/makefile.cpp
...file modified
include/CUC/makefile.cpp
...file modified
include/DCW/makefile.cpp
...file modified
include/FF/makefile.cpp
...file modified
include/FW/makefile.cpp
...file modified
src/makefile.cpp
...file modified
src/AA/makefile.cpp
...file modified
src/AA/DCW/makefile.cpp
...file modified
src/AA/VPF/makefile.cpp
...file modified
src/AA/freeform/makefile.cpp
...file modified
src/AA/generic/makefile.cpp
...file modified
src/CBP/makefile.cpp
...file modified
src/CSC/makefile.cpp
...file modified
src/CUC/makefile.cpp
...file modified
src/CUC/ODL/makefile.cpp
...file modified
src/CUC/UDUNITS/makefile.cpp
...file modified
src/EPH/makefile.cpp
...file modified
src/EPH/orbsim/makefile.cpp
...file modified
src/EPH/gbadsim/makefile.cpp
...file modified
src/GCT/makefile.cpp
...file modified
src/GCT/freeware/makefile.cpp
...file modified
src/IO/makefile.cpp
...file modified
src/IO/GEN/makefile.cpp
...file modified
src/IO/L0/makefile.cpp
...file modified
src/IO/L0/L0sim/makefile.cpp
...file modified
src/MEM/makefile.cpp
...file modified
src/MET/makefile.cpp
...file modified
src/MET/support/makefile.cpp
...file modified
src/MET/tools/makefile.cpp
...file modified
src/PC/makefile.cpp
...file modified
src/SMF/makefile.cpp
...file modified
src/TD/makefile.cpp
...file modified
src/DEM/makefile.cpp
...file modified
src/TSF/makefile.cpp
...file modified
src/XML/makefile.cpp
...file modified
src/XML/xml/Makefile.cpp
...file modified
src/XML/xsl/Makefile.cpp
...file modified
message/makefile.cpp
...file modified

Setting ZLIB home directory to: /seadas/lcl/ocssw/opt

Setting JPEG home directory to: /seadas/lcl/ocssw/opt

Setting SZIP home directory to: /seadas/lcl/ocssw/opt
Setting LD_LIBRARY_PATH to /seadas/lcl/ocssw/opt/lib

Setting HDF home directory to: /seadas/lcl/ocssw/opt

***** Copying JPEG, ZLIB and SZIP header and library *********

********** files to relevant directories in HDF.    **********

Setting HDF5 home directory to: /seadas/lcl/ocssw/opt
Adding /seadas/lcl/ocssw/opt/lib to LD_LIBRARY_PATH

Setting HDFEOS home directory to: /seadas/lcl/ocssw/opt/EOS

Setting HDFEOS5 home directory to: /seadas/lcl/ocssw/opt/EOS

Running the Toolkit Installation Script ...

Running the Toolkit Installation Script without AA Tool.
bin/INSTALL: 3190: bin/INSTALL: /seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit: not found
bin/INSTALL: 3192: bin/INSTALL: cannot create /seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin/linux64/pgs-env.csh: Directory nonexistent

INSTALL: Error: The Toolkit Installation Script failed or had errors.
SDP Toolkit installation cancelled
***** INSTALL failed *****

What is the AA tool?  Is that a part of ubuntu that I need to install or something else?

bruce
Posts: 85
Joined: Thu Mar 17, 2005 4:36 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by bruce » Tue Jun 19, 2018 10:10 am America/New_York

but the first file that's "not found" appears to be there...

bcb@ubuntu:/seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin$ ls
common        INSTALL-HDF4.0r2  INSTALL-HDF4.2r10  INSTALL-HDF4.X      INSTALL-HDF5-1.6.2   INSTALL-HDF5-1.8.12  INSTALL-JPEGSRC.v6b  INSTALL-ZLIB   pgs-flags
ftp.csh        INSTALL-HDF4.1r1  INSTALL-HDF4.2r2   INSTALL-HDF5       INSTALL-HDF5-1.6.3   INSTALL-HDF5-1.8.3  INSTALL-SZIP       INSTALL-ZLIB-1.2.1  pgs-flags-cpp
INSTALL        INSTALL-HDF4.1r3  INSTALL-HDF4.2r3   INSTALL-HDF5-1.4.0  INSTALL-HDF5-1.6.4   INSTALL-HDF5-1.8.5  INSTALL-SZIP1.1      INSTALL-ZLIB-1.2.3  tmp
INSTALL-4.1r3_aix   INSTALL-HDF4.1r4  INSTALL-HDF4.2r4   INSTALL-HDF5-1.4.1  INSTALL-HDF5-1.6.5   INSTALL-HDF5-1.8.8  INSTALL-SZIP1.2      INSTALL-ZLIB-1.2.5
INSTALL-AAdata      INSTALL-HDF4.1r5  INSTALL-HDF4.2r5   INSTALL-HDF5-1.4.3  INSTALL-HDF5-1.6.6   INSTALL-HDFEOS5-Wrap  INSTALL-SZIP2.0      INSTALL-ZLIB-1.2.8
INSTALL-HDF      INSTALL-HDF4.2r0  INSTALL-HDF4.2r6   INSTALL-HDF5-1.4.4  INSTALL-HDF5-1.6.7   INSTALL-HDFEOS-Wrap  INSTALL-SZIP2.1      mkpgslib
INSTALL-HDF4.0r1p1  INSTALL-HDF4.2r1  INSTALL-HDF4.2r9   INSTALL-HDF5-1.6.1  INSTALL-HDF5-1.8.11  INSTALL-JPEG    INSTALL-Toolkit      mkpgslib.cpp

gfireman
Posts: 64
Joined: Thu Jan 07, 2010 2:59 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by gfireman » Tue Jun 19, 2018 2:35 pm America/New_York

The AA tool is a part of the Science Data Toolkit that we don't use, so that's not the problem.

This is really bizarre.  Is /seadas/lcl on a remote disk, that maybe just glitched out for a second?
Please try:
ls -l $LIB3_DIR/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit
Mine looks like:
-rwxr-xr-x 1 gfireman gfireman 177704 2013-12-12 11:59 /accounts/gfireman/ocssw/opt/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit

Which shell are you using?

bruce
Posts: 85
Joined: Thu Mar 17, 2005 4:36 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by bruce » Tue Jun 19, 2018 3:02 pm America/New_York

bcb@modis:~$ ls -l $LIB3_DIR/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit
-rwxr-xr-x 1 bcb bcb 177265 Jun 19 10:06 /seadas/lcl/ocssw/opt/src/sdptk/TOOLKIT/bin/INSTALL-Toolkit
bcb@modis:~$

bash, it's the only shell worth using (I know them's fightin words to some :-)

It's a local disk, and the failure is repeatable

Everything else that comes after it in the buildit sequence (other than mapi and sdst which appear to depend on it) build just fine.

gfireman
Posts: 64
Joined: Thu Jan 07, 2010 2:59 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by gfireman » Tue Jun 19, 2018 3:28 pm America/New_York

I use bash too :)
Your file is post-patch, which accounts for the different date and size.

The ECS tools page lists a newer version of the SDPTK, but I don't know if it will work any better.  We are still running Ubuntu 16.04, so I don't have a way to test either installation.  (Don could, but he's been called away this week).

The hdfeos, hdfeos5, sdptk, mapi, and sdst libraries are used only in generating MODIS L1A, GEO, and L1B.  If you don't need those, we should be able to come up with a way to compile the rest of OCSSW without them.

bruce
Posts: 85
Joined: Thu Mar 17, 2005 4:36 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by bruce » Tue Jun 19, 2018 3:44 pm America/New_York

For the time being, I'm mostly interested in mucking around in l2gen, so as long as I can rebuild that, I should be happy (at least for a little while :-) )

gfireman
Posts: 64
Joined: Thu Jan 07, 2010 2:59 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by gfireman » Tue Jun 19, 2018 5:17 pm America/New_York

I believe the following will work.  It compiles, but I haven't tested it.

1. In file $LIB3_DIR/src/BuildIt
remove from the subdirs list:  hdfeos hdfeos5 sdptk mapi sdst

2. Delete from directory $OCSSWROOT/cmake/Modules
FindHDFEOS.cmake
FindHDFEOS5.cmake
FindMapi.cmake
FindPGSTK.cmake
FindSDST.cmake

3. In file $OCSSWROOT/src/CMakeLists.txt
delete or comment out all the "add_subdirectory" lines in the "MODIS-specific" section.

bruce
Posts: 85
Joined: Thu Mar 17, 2005 4:36 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by bruce » Wed Jun 20, 2018 9:05 am America/New_York

presuming the next steps are still 

cd build
cmake ..
make -j 20


It runs for a while and then errors out as in the attached log file
attachment 1

gfireman
Posts: 64
Joined: Thu Jan 07, 2010 2:59 pm America/New_York
Answers: 0

Building on Ubuntu 18.04

by gfireman » Wed Jun 20, 2018 10:20 am America/New_York

After making those changes, I'd start fresh: delete the existing build directory, then follow the build instructions in README.md, omitting the download steps.
I'd also use just "make" instead of "make -j 20".   Leaving out the "-j" parameter will run only one job:  slower, but it's easier to read a linear log when there's trouble.

Turns out we do have an Ubuntu 18.04 system; I'll see if I can get the standard compile to work there.

Post Reply