atteph failing

oo_processing · by **oo_processing** » Fri Feb 14, 2020 1:38 pm America/New_York

I had had several modis aqua passes fail in the same way.

To determine what atteph files I need in advance (as the -d must be used now) I run:
searobin 870 :> modis_atteph.py -d MOD00.P2020045.0620.CAPE_COD_FULL.L1A_LAC
Aqua
att1=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0615.061
att2=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0620.061
att3=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0630.061
att4=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0625.061
eph1=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0615.061
eph2=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0620.061
eph3=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0625.061
eph4=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0630.061

Of course the att1-att4 and eph1-eph4 need to be changed to a1 e1 etc.
However, the modis_GEO.py doesn't accept keywords a|e 3|4.
Traceback (most recent call last):
File "/optics1/software/seadas/seadas-7.5/ocssw/scripts/modis_GEO.py", line 187, in <module>
sys.exit(main())
File "/optics1/software/seadas/seadas-7.5/ocssw/scripts/modis_GEO.py", line 170, in main
timeout=timeout)
File "/optics1/software/seadas/seadas-7.5/ocssw/scripts/modules/modis_GEO_utils.py", line 73, in __init__
if not self[param]:
File "/optics1/software/seadas/seadas-7.5/ocssw/scripts/modules/modis_GEO_utils.py", line 80, in __getitem__
return self.__dict__[index]
KeyError: 'a3'

So why does modis_atteph.py return them?
I tried to use a -s -e as well (and fixed for the att and eph tags)

a1=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0610.061
a2=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0615.061
e1=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0615.061
e2=/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0610.061

For some reason neither produce a valid GEO

modis_GEO.py --parfile=./2020045061501.atteph.test MOD00.P2020045.0620.CAPE_COD_FULL.L1A_LAC -o MOD00.P2020045.0620.CAPE_COD_FULL.GEO --threshold=95 --ancdb=./ancillary_data.db
./2020045061501.atteph.test
{'geogen': {'a1': '/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0610.061', 'a2': '/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0615.061', 'e1': '/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0615.061', 'e2': '/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0610.061'}}
/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0610.061
/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1ATTNR_NRT.A2020045.0615.061
/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0615.061
/optics1/software/seadas/seadas-7.5/ocssw/var/anc/2020/045/PM1EPHND_NRT.A2020045.0610.061
MODIS GEO version 6.1.0, built Nov 17 2018 15:44:32
scan: 0 out of 399 Fri Feb 14 13:26:09 2020
scan: 10 out of 399 Fri Feb 14 13:26:09 2020
scan: 20 out of 399 Fri Feb 14 13:26:10 2020
scan: 30 out of 399 Fri Feb 14 13:26:10 2020
scan: 40 out of 399 Fri Feb 14 13:26:10 2020
scan: 50 out of 399 Fri Feb 14 13:26:10 2020
scan: 60 out of 399 Fri Feb 14 13:26:10 2020
scan: 70 out of 399 Fri Feb 14 13:26:10 2020
scan: 80 out of 399 Fri Feb 14 13:26:11 2020
scan: 90 out of 399 Fri Feb 14 13:26:11 2020
scan: 100 out of 399 Fri Feb 14 13:26:11 2020
scan: 110 out of 399 Fri Feb 14 13:26:11 2020
scan: 120 out of 399 Fri Feb 14 13:26:11 2020
scan: 130 out of 399 Fri Feb 14 13:26:11 2020
scan: 140 out of 399 Fri Feb 14 13:26:11 2020
scan: 150 out of 399 Fri Feb 14 13:26:12 2020
scan: 160 out of 399 Fri Feb 14 13:26:12 2020
scan: 170 out of 399 Fri Feb 14 13:26:12 2020
scan: 180 out of 399 Fri Feb 14 13:26:12 2020
scan: 190 out of 399 Fri Feb 14 13:26:12 2020
scan: 200 out of 399 Fri Feb 14 13:26:12 2020
scan: 210 out of 399 Fri Feb 14 13:26:12 2020
scan: 220 out of 399 Fri Feb 14 13:26:13 2020
scan: 230 out of 399 Fri Feb 14 13:26:13 2020
scan: 240 out of 399 Fri Feb 14 13:26:13 2020
scan: 250 out of 399 Fri Feb 14 13:26:13 2020
scan: 260 out of 399 Fri Feb 14 13:26:13 2020
scan: 270 out of 399 Fri Feb 14 13:26:13 2020
scan: 280 out of 399 Fri Feb 14 13:26:13 2020
scan: 290 out of 399 Fri Feb 14 13:26:13 2020
scan: 300 out of 399 Fri Feb 14 13:26:14 2020
scan: 310 out of 399 Fri Feb 14 13:26:14 2020
scan: 320 out of 399 Fri Feb 14 13:26:14 2020
scan: 330 out of 399 Fri Feb 14 13:26:14 2020
scan: 340 out of 399 Fri Feb 14 13:26:14 2020
scan: 350 out of 399 Fri Feb 14 13:26:14 2020
scan: 360 out of 399 Fri Feb 14 13:26:14 2020
scan: 370 out of 399 Fri Feb 14 13:26:14 2020
scan: 380 out of 399 Fri Feb 14 13:26:15 2020
scan: 390 out of 399 Fri Feb 14 13:26:15 2020
Percent valid data (0.00) is less than threshold (95.00)
ERROR: MODIS geolocation processing failed.

gfireman · by **gfireman** » Fri Feb 14, 2020 2:28 pm America/New_York

Brock,

This failure is related to this post. If my fix to allow 3 atteph files has not yet been released, you can download the new files from this forum post. We do not anticipate ever being able to use four atteph files.

There may be a bug in the way the parfile is parsed; we'll look into it. In the meantime, you could do something like:

atteph=`grep "[123]=" $l1afile.atteph | sed 's/^/--/' | tr "\n" " "`
modis_GEO.py $atteph $l1afile -o $geofile --threshold=95

Hope that helps,
Gwyn

by **OB.DAAC - SeanBailey** » Fri Feb 14, 2020 3:52 pm America/New_York

Brock,

The fix Gwyn mentioned is in the latest update. The modis_atteph.py returns all that match the time period requested. In this case, that means 4 sets since the NRT att/eph are in play.
The geolocation code, however, only supports 3 sets. If we limited the result of modis_atteph.py to 3 sets, it'd be lying to you. If we modified the geolcation script to only use (at most) 3 sets, you'd get invalid geolcation where the data aren't covered by the att/eph inputs. This issue stems from a decision made long ago (upstream of us) to chunk the NRT att/eph files into 5 minute granules to match the L1A granule size. It's my long held opinion that this was a bad decision. You are seeing the consequences of it.

If you wait until the refined att/eph files are available, there will only be at most two sets that match and the geolocation process will be happy. Or you could limit the size of the granule you process to require no more than 3 sets.

Regards,
Sean