l2gen open error with :Problem creating product XML file:l2gen returned nonzero exitvalue

[halomarine]

hello
because of bad network, I installed seadas7.5.3 and ocssw offline in ubuntu 18.04 linux with install_ocssw.py

later, it was found problems with opening l2gen:

ERROR: Problem creating product XML file:l2gen returned nonzero exitvalue.

I hope to process MODIS AQUA and MODIS TERRA data and obtain Rrc data through l2gen.

I've seen a lot of posts with the same problem, but I still don't understand some of solutions:

"SeaDAS needs to be able to write to the $HOME/.seadas/l2gen directory in order for the l2gen GUI to initialize.
Make sure the this directory and specifically the file productInfo.xml  have read/write permissions for the user. ——by Sean"

or “The error "ERROR: Problem creating product XML file: l2gen returned nonzero exitvalue" can also occur if there are permission problems or missing files in the $OCSSWROOT/run/data directories, or if an incorrect value is used for $OCSSWROOT. ——by gnwiii”

Could you please provide me with more detailed operation steps?

thanks a lot

[gnwiii]

Offline installs assume some basic understanding of the linux command-line.   The most common reason for permission problems is incorrect application of the sudo command, resulting in files and directories owned by the linux "root" user.  

In the last response to Error initializing l2gen and l3gen (SEVERE: productInfo.xml not found!) I mention the benefits of spending half a day to learn some basics of the linux command-line.   This post has a link to a book on
the linux command line.  Chapter 9 explains linux permissions and the sudo command. 

I should add that linux started as a user driven project with contributions from many users.   Although most linux development is now done in large corporations, the cooperative spirit is still very much alive.  If you have a friend or colleague who is familiar with linux they will generally be eager to help with things you don't understand.

[halomarine]

thanks a lot!
I'm sorry to convey the failed message to you, I still haven't solved this problem.

I remember that I didn't use sudo command.

I learned from chapter 9 that there was a hidden folder:eek:, and then found details with the ls-al command as follows:
drwxr-xr-x 6 jinchao jinchao 4096 2?  11 17:16 .seadas
drwxr-xr-x 9 jinchao jinchao 4096 2?  11 17:43 seadas

and then I changed the permissions of two files, but still didn't work.
chmod -r 777 / home/jinchao/seadas
chmod -r 777 .seadas

Enclosed is my new bash.bashrc
export SEADAS=/home/jinchao/seadas
export PATH=$SEADAS/bin:$PATH
export OCSSWROOT=/home/jinchao/seadas/ocssw
source $OCSSWROOT/OCSSW_bash.env  (I saw a file I had downloaded named OCSSW_bash.env.v7.5)

I picked up all similar posts before, reinstall seadas and OCSSW for many times but still failed to open l2gen,
I looked through a lot of previous posts and tried a lot of methods, but nothing worked out.In great distress.:cry:
thanks a lot!
jinc

[gnwiii]

Most problems with linux have a simple explanation.  It is rarely necessary to do the Windows uninstall/reinstall dance because there aren't any hidden configuration details.  Finding the simple explanation is, however, sometimes a bit challenging.   I assume you are trying to run l2gen using the SeaDAS GUI OCSSW menu.   The error you get generally means l2gen isn't able to run.  It may have been misplaced, or some files it requires were misplaced.

It looks like your permissions were OK, so changing to 777 wasn't necessary.   You have: export OCSSWROOT=/home/jinchao/seadas/ocssw, but the default for install_ocssw.py is to install the OCSSW software directly in your HOME directory.   To check, you can use: "ls -ld ~/ocssw ~/seadas/ocssw".   You appear to have ~/seadas/ocssw, but the installation there may be incomplete.    Have you tried running "Update data processors" from the GUI OCSSW menu?  

If you have installed the OCSSW software in ~/ocssw and want to use that, you will need to use: export OCSSWROOT=/home/jinchao/ocssw.  If you want to run the OCSSW programs from the SeaDAS GUI, you also have to change the location in the ~/seadas/config/seadas.conf.  Open this file in the text editor and look near the bottom of the file for the line:

seadas.ocssw.root = ${seadas.home}/ocssw

Just edit this line to read:

seadas.ocssw.root = ${seadas.home}/../ocssw

If you prefer to stick with the ~/seadas/ocssw directory you can try running install_ocssw.py --install-dir=~/seadas/ocssw ... where the ... is shorthand for the options you used previously to select the GIT branch, bundles location, and sensors.  This should ensure that the  ~/seadas/ocssw installation is complete.  

When running install_ocssw.py I often encounter network problems in the final update LUTs step.  You don't need to run install_ocssw.py again -- you can just run the update LUTs manually, either from the GUI or on the lnux bash command line.

[halomarine]

Thank you for your detailed explanation.

I also want to show my installation process in detail:

the installation words:./install_ocssw.py   -v   -i /home/jinchao/seadas/ocssw   -l /home/jinchao/new   -b v7.5   -a linux_64   --terra   --aqua   --direct-broadcast

I have two computers in China, one is Linux system and the other is Windows. I don't know how to use VPN in Linux. Only by using VPN on the Windows computer can I download and install necessary files and communicate with you.Transfer the download to a Linux computer and install it manually.VPNS are not strong.I don't know how many times I downloaded it before it was complete.So i don't want to download again.

And you can see from the way I deal with it that one problem after another is driving me crazy.:grin:

Each time I open l2gen ,show  problem creating product xml file; and I click the run button, see:

/home/jinchao/seadas/ocssw/scripts/ocssw_runner:line 23 :l2gen:no command found;

I opened ocssw_runner, found nothing but "$*" in line 23.

Is it really the reason for the incomplete installation? Is it possible that one of the downloaded files was incomplete? I am scared about downloading all the bundles again:cry::cry:

How can we solve this problem? What makes me stuck here?

sincerely thank you for your help!
jinc

[gnwiii]

I can appreciate your difficulties with the lack of internet access.   Scientists in developing countries or on ships at sea also have very limited (e.g., slow, erratic, and or expensive)  internet access so also need to use the bundles. 

Unfortunately, the bundles are not usually updated with recent changes.   The install_ocssw.py script uses git to update the installation with the most recent changes.   These changes are generally small so can be done with a slow and/or erratic internet connection.   Internet access is also required to download ancillary files needed by l2gen.   For systems that have no internet access at all it is possbile to download ancillary files on another system and copy them to the system with the OCSSW software, but you may also need to get recent software updates.  The most recent updates were, however, changes needed to download data so don't apply to systems without internet access.  

According to the install_ocssw.py command line, your OCSSW software was installed to ~/seadas/ocssw, but the l2gen program seems to be missing.  To confirm, what do you get from "ls -l $OCSSWROOT/bin/l2gen" and "ls -1 $OCSSWROOT/bin/* | wc -l"?   Here:

$ ls -l $OCSSWROOT/bin/l2gen
-rwxr-xr-x. 1 seadas seadas 4418424 Sep 24 08:23 /home/seadas/ocssw/bin/l2gen
$ ls -1 $OCSSWROOT/bin/* | wc -l
68

Your l2gen should be the same size, but the date will be the installation date.  The second command counts the number of files, so should be the same on both systems.

If l2gen is missing, you can use the git bundle (bin-linux_64.bundle) without running install_ocssw.py, as described in Ocean Data Science Software Repositories.

[halomarine]

It shows:
-rwxrwxrwx 1 jinchao jinchao 4418424 Feb 11 21:55 /home/jinchao/seadas/ocssw/bin/l2gen

68

There doesn't seem to be anything missing

[gnwiii]

You original post referred to two suggestions:

Sean: permission problems for ~/.seadas/l2gen -- thanks to the linuxcommand book you should be able to check these.

Me: missing runtime data files -- due to the confusion over the changes to the directory structure and the versions of the bundle files this possiblity has to be considered.
One simple thing to check is the directory tree structure inside the ocssw directory using the tree command:

tree -d -H "file:/$OCSSWROOT" $OCSSWROOT > ocsswtree.html

See the attached html file.   This will vary depending on the configured sensors, etc., but the basic structure should be similar across installations.

The next step is to see if there is an error running l2gen outside the GUI.   In a terminal:
$ cd $OCSSWROOT/bin
$ ./l2gen  |head -1
l2gen 9.5.0-V2019.3 (Aug 16 2019 12:47:35)

If this works, you can try to run the command to generate the XML file that failed using the GUI:

$ ./l2gen prodxmlfile=~/.seadas/l2gen/productInfo.xml
Writing product information to XML file /home/seadas/.seadas/l2gen/productInfo.xml

If this gives an error there may be some indication of the reason it failed.
attachment 1

[halomarine]

Through the first step, I get one of my directory trees, which, from the bottom version, shows?

Tree v1.7.0c 1996-2014 by Steve Baker and Thomas Moore;

The HTML file you gave me is newer than mine

Tree v1.8.0 c 1996-2018 by Steve Baker and Thomas Moore;

I think there might be a version problem here.

Then I followed the second step and found

The./l2gen | head-1 command is fine, no error;

the $. / l2gen prodxmlfile = ~ /. Seadas l2gen/productInfo. XML  command gives an error:

/home/jinchao/seadas/ocssw/oel_util/libpiutils/productInfo.cpp Line 268: could not open product XML file=/home/jinchao/seadas/ocssw/run/data/common/product.xml File was not found

here I found the product.xml file is in :/home/jinchao/seadas/ocssw/share/common instead of  /home/jinchao/seadas/ocssw/run/data/common

I also realize when I do source $OCSSW/OCSSW_bash.env

I do printenv |grep OC

found that OCDATAROOT=/home/jinchao/seadas/ocssw/run/data
OCVARROOT=/home/jinchao/seadas/ocssw/run/var

I actually don't have the run directory..

[gnwiii]

The html version doesn't matter, what is important is the directory structure.  Older OCSSW versions used "OCDATAROOT=$OCSSWROOT/run/data", now it is
"OCDATAROOT=$OCSSWROOT/share", so you ended up with an old version of OCSSW_bash.env.   Try the  current v7.5 version.