viirsn processing after SeaDAS 7.4 upgrade

Please enter here to ask a question about any NASA Science related topics!
jcoca
Posts: 12
Joined: Thu Feb 11, 2021 6:51 am America/New_York

viirsn processing after SeaDAS 7.4 upgrade

by jcoca » Fri Mar 10, 2017 5:26 am America/New_York

Hello

After installing SeaDAS 7.4 we are experiencing problems with VIIRSN processing (Also reprojecting files with gpt.sh, solved thanks to the aynur response to the alakes post). After the instalation, we have run  update_luts.py for viirsn, aqua and terra.

- Example of the output of geolocate_viirs:

Main:: Geolocating L1A V2017068033600.L1A_SNPP.nc

VcstCmnGeoNetcdfInputItem:: Failure opening LUT file: /seadas/ocssw/run/var/viirsn/cal/OPER/VIIRS_NPP_CMN_LUT_v1.1.0.1_vcst.nc
VcstCmnGeoNetcdfInputItem:: Failure reading netCDF file for LUT JPL_PLANET_EPHEM
VcstCmnGeo::Unable to read JPL.
Main:: Geolocation processing failure for ifile=V2017068033600.L1A_SNPP.nc

- The content for viirsn instruments_default.par file, do not match the directory tree/content of the seadas installation:

static_lut_file=$OCVARROOT/viirsn/cal/OPER/VIIRS_NPP_CAL_STATIC_LUT_v1.1.0.1_obpg.nc

rsb_dynamic_lut_file=$OCVARROOT/viirsn/cal/OPER/VIIRS_NPP_CAL_RSB_DYNAMIC_LUT_v1.1.0.1_obpg.nc
dnb_dynamic_lut_file=$OCVARROOT/viirsn/cal/OPER/VIIRS_NPP_CAL_DNB_DYNAMIC_LUT_v1.1.0.1_vcst.nc

straylight_lut_file=$OCVARROOT/viirsn/cal/OPER/VIIRS_NPP_CAL_STRAYLIGHT_DYNAMIC_LUT_v1.1.0.1_vcst.nc

cmn_lut_file=$OCVARROOT/viirsn/cal/OPER/VIIRS_NPP_CMN_LUT_v1.1.0.1_vcst.nc
geo_lut_file=$OCVARROOT/viirsn/cal/OPER/VIIRS_NPP_GEO_LUT_v1.1.0.1_obpg.nc

polar_wander_file=$OCVARROOT/viirsn/polar_wander.ascii
leapsec_path=$OCVARROOT/viirsn/

terrain_path=$OCDATAROOT/viirsn/dem/

- content of /seadas/ocssw/run/var/viirsn/cal/OPER directory:

VIIRS_NPP_CAL_DNB_DYNAMIC_LUT_v2.0.0.2_obpg.nc
VIIRS_NPP_CAL_RSB_DYNAMIC_LUT_v2.0.0.2_obpg.nc
VIIRS_NPP_CAL_STATIC_LUT_v2.0.0.2_obpg.nc
VIIRS_NPP_CAL_STRAYLIGHT_DYNAMIC_LUT_v2.0.0.2_obpg.nc
VIIRS_NPP_CMN_LUT_v2.0.0.2_obpg.nc
VIIRS_NPP_GEO_LUT_v2.0.0.2_obpg.nc
VIIRS-SDR-F-LUT_npp_OBPG20140716_v3.h5

- terrain_path=$OCDATAROOT/viirsn/dem/ dem subdirectory do not exists in our SeaDAS 7.4 installation

- The corresponding environmental variables correctly set:

echo $OCVARROOT
/seadas/ocssw/run/var
echo $OCDATAROOT
/seadas/ocssw/run/data

- The linux operating system version:
LSB Version: core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noarch
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

We experienced the same problem in two computers, that have the same tree seadas installation and operating system. MODIS processing is working fine.

Regards
Josep

Tags:

oo_processing
Posts: 211
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

viirsn processing after SeaDAS 7.4 upgrade

by oo_processing » Fri Mar 10, 2017 11:31 am America/New_York

I had the same issue, I just posted here, but it not just seadas 7.4. I am still using seadas 7.3, and it was an automated LUT update that changed the files last night. Neither 7.3 or 7.4 work now.

https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=6521

gnwiii
Posts: 641
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

viirsn processing after SeaDAS 7.4 upgrade

by gnwiii » Fri Mar 10, 2017 1:57 pm America/New_York

I have SeaDAS 7.4 on Linux Mint 18.1 (based on Ubuntu 16.04).  After running update_luts.py I also have the new $OCVARROOT/viirsn/cal/OPER/*_LUT_v2.0.0.2_obpg.nc files but ocssw/run/data/viirsn/instrument_defaults.par still has *_LUT_v1.1.0.1_obpg.nc.

oo_processing
Posts: 211
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

viirsn processing after SeaDAS 7.4 upgrade

by oo_processing » Fri Mar 10, 2017 2:24 pm America/New_York

That is what here:
https://oceandata.sci.gsfc.nasa.gov/Ancillary/LUTs/viirsn/cal/OPER/
But as you said, not ./ocssw/run/data/viirsn/instrument_defaults.par

It would be simple to hack the instrument_defaults par by replacing the - with the + as per the LUT update, but I have to wonder if the new files we placed in error?

- OPER:VIIRS_NPP_CMN_LUT_v1.1.0.1_vcst.nc
+ OPER:VIIRS_NPP_CMN_LUT_v2.0.0.2_obpg.nc

- OPER:VIIRS_NPP_CAL_RSB_DYNAMIC_LUT_v1.1.0.1_obpg.nc
+ OPER:VIIRS_NPP_CAL_RSB_DYNAMIC_LUT_v2.0.0.2_obpg.nc

- OPER:VIIRS_NPP_CAL_STRAYLIGHT_DYNAMIC_LUT_v1.1.0.1_vcst.nc
+ OPER:VIIRS_NPP_CAL_STRAYLIGHT_DYNAMIC_LUT_v2.0.0.2_obpg.nc

- OPER:VIIRS_NPP_GEO_LUT_v1.1.0.1_obpg.nc
+ OPER:VIIRS_NPP_GEO_LUT_v2.0.0.2_obpg.nc

- OPER:VIIRS_NPP_CAL_STATIC_LUT_v1.1.0.1_obpg.nc
+ OPER:VIIRS_NPP_CAL_STATIC_LUT_v2.0.0.2_obpg.nc

- OPER:VIIRS_NPP_CAL_DNB_DYNAMIC_LUT_v1.1.0.1_vcst.nc
+ OPER:VIIRS_NPP_CAL_DNB_DYNAMIC_LUT_v2.0.0.2_obpg.nc

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1219
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1

viirsn processing after SeaDAS 7.4 upgrade

by OB.DAAC - SeanBailey » Fri Mar 10, 2017 3:20 pm America/New_York

As Brock noted this was not related to the v7.4 update.  It's a long and boring story, but the synopsis is this.  We've updated the VIIRS processing code in our production environment to a new version, however the "var" tree is shared and as the Highlander knows, there can be only one.  I've pushed updated versions of the affected binaries and updated the VIIRS defaults files to the v7.3 and v7.4 branches.  If you update your OCSSW code, you should be once again able to process VIIRS.

Regards,
Sean

oo_processing
Posts: 211
Joined: Wed Apr 06, 2005 12:11 pm America/New_York

viirsn processing after SeaDAS 7.4 upgrade

by oo_processing » Fri Mar 10, 2017 3:24 pm America/New_York

Yes, that works.... of course :grin:
There can be only one, and it appears that be you :wink:

And as a reminder for those working with 7.3, you will need to setup your environment and run:
./install_ocssw.py --viirsn
Because the "Update OC Processers" doesn't work in 7.3

Cheers!

jcoca
Posts: 12
Joined: Thu Feb 11, 2021 6:51 am America/New_York

viirsn processing after SeaDAS 7.4 upgrade

by jcoca » Sun Mar 12, 2017 1:43 pm America/New_York

Hello

I have updated the ocssw, however there is still a problem related to terrain_path=$OCDATAROOT/viirsn/dem/ dem subdirectory.
The dem subdirectory is still missing. Are dem data placed in other location?

The ouput of geolocate_viirs:

geolocate_viirs ifile=V2017068132400.L1A_SNPP.nc

Main:: Geolocating L1A V2017068132400.L1A_SNPP.nc
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.                                                                  
                                                                                                                                                      
VcstViirsGeo:: Failure creating GEO MOD file:                                                                                                        
                                                                                                                                                      
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.                                                                  
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.                                                                  
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.                                                                  
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
Unable to open directory (/seadas/ocssw/run/data/viirsn/dem/) for reading DEM files.
terminate called after throwing an instance of 'netCDF::exceptions::NcBadId'
  what():  NcBadId: Not a netcdf id
file: ncVar.cpp  line:998
Abortado (`core' generado)

Josep

gnwiii
Posts: 641
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

viirsn processing after SeaDAS 7.4 upgrade

by gnwiii » Sun Mar 12, 2017 5:39 pm America/New_York

If you want to do VIIRS geolocation you need to install the files manually, see What is needed to geolocate NASA L1 VIIRS data?-SeaDAS 7 FAQ

jcoca
Posts: 12
Joined: Thu Feb 11, 2021 6:51 am America/New_York

viirsn processing after SeaDAS 7.4 upgrade

by jcoca » Tue Mar 14, 2017 5:20 am America/New_York

Hello

I have dowloaded and extracted the   file DEM-LWM.tar.gz (ftp://samoa.gsfc.nasa.gov/pub/VIIRSL1/) in my $OCDATAROOT/viirsn directory. The dem/ and lw/ directories have been created. However I am still experiencing problems. This is the current output of geolocate_viirs:

geolocate_viirs ifile=V2017072153000.L1A_SNPP.nc

Main:: Geolocating L1A V2017072153000.L1A_SNPP.nc

VcstViirsGeo:: Failure creating GEO MOD file:

terminate called after throwing an instance of 'netCDF::exceptions::NcBadId'
  what():  NcBadId: Not a netcdf id
file: ncVar.cpp  line:998
Abortado (`core' generado)

Josep

gnwiii
Posts: 641
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

viirsn processing after SeaDAS 7.4 upgrade

by gnwiii » Tue Mar 14, 2017 8:44 am America/New_York

I get the same error on 2 different systems (macOS and linux).  Using "strace geolocate_viirs verbose=true ifile=V2017072153000.L1A_SNPP.nc" reveals that the problem is with "$OCDATAROOT/viirsn/dem//NPP_VIIRS_Terrain-Eco-ANC-Tile_N0722_v1.5.04.00_LP.teb".   I have "geolocate_viirs 2.0 (Mar 10 2017 13:52:39)".  Maybe this new version wants the advertised smaller dem files.

Post Reply