Page 1 of 1

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 3:08 pm America/New_York
by oo_processing
[bmurch@optics0 S4P_MODIS_H5_MODIST.TWS.2021.029_version_0.10_nVa498i1_TEST]$ modis_L1A.py --verbose ./MOD00.A2021029.1350_1.PDS -o ./MOD00.A2021029.1350.TWS_FULL.L1A_LAC --startnudge=0 --stopnudge=10
Determining pass start and stop time...

Using specified output L1A filename: ./MOD00.A2021029.1350.TWS_FULL.L1A_LAC
Traceback (most recent call last):
  File "/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/scripts/modis_L1A.py", line 92, in <module>
    m.l0()
  File "/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/scripts/modules/modis_L1A_utils.py", line 241, in l0
    self.gransec = ProcUtils.diffsecs(self.start, self.stop, 't')
  File "/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/scripts/modules/ProcUtils.py", line 370, in diffsecs
    return (t1-t0).total_seconds()
AttributeError: 'datetime.timedelta' object has no attribute 'total_seconds'

modis_L1A.py --verbose ./MOD00.P2021034.1755.PDS -o ./MOD00.P2021034.1755.SE_FL_FULL.L1A_LAC --startnudge=0 --stopnudge=10
Determining pass start and stop time...                                                                                                                                                                                      

Using specified output L1A filename: ./MOD00.P2021034.1755.SE_FL_FULL.L1A_LAC
Traceback (most recent call last):                                          
  File "/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/scripts/modis_L1A.py", line 92, in <module>
    m.l0()                                                                                          
  File "/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/scripts/modules/modis_L1A_utils.py", line 241, in l0
    self.gransec = ProcUtils.diffsecs(self.start, self.stop, 't')                                            
  File "/shares/cms_optics/apps/seadas/seadas-7.5.3/ocssw/scripts/modules/ProcUtils.py", line 370, in diffsecs
    return (t1-t0).total_seconds()                                                                           
AttributeError: 'datetime.timedelta' object has no attribute 'total_seconds'

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 3:52 pm America/New_York
by OB.DAAC - SeanBailey
AttributeError: 'datetime.timedelta' object has no attribute 'total_seconds'

That's nothing to do with our code, that's an error you get if you're running a seriously old version of python...
What python version are you running this with?

Sean

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 4:05 pm America/New_York
by oo_processing
Sean,

My bad. I didin't change my env when I was trying to replicate the automated process on the command line.
I did find some bad anc data but I fixed it:
$ file /shares/cms_optics/apps/seadas/anc/2021/034/*
/shares/cms_optics/apps/seadas/anc/2021/034/AM1ATTN0.A2021034.0000.002:     data                                                  
...                                                 
/shares/cms_optics/apps/seadas/anc/2021/034/AM1EPHN0.A2021034.1200.002:     data                                                  
/shares/cms_optics/apps/seadas/anc/2021/034/N202103400_MET_NCEP_6h.hdf:     Hierarchical Data Format (version 4) data             
/shares/cms_optics/apps/seadas/anc/2021/034/N202103406_MET_NCEP_6h.hdf:     Hierarchical Data Format (version 4) data             
/shares/cms_optics/apps/seadas/anc/2021/034/N202103412_MET_NCEP_6h.hdf:     Hierarchical Data Format (version 4) data             
/shares/cms_optics/apps/seadas/anc/2021/034/PM1ATTNR_NRT.A2021034.0110.061: data                                                  
...
/shares/cms_optics/apps/seadas/anc/2021/034/PM1EPHND_NRT.A2021034.1805.061: data

$ modis_L1A.py  ./MOD00.A2021034.0415.PDS -o ./MOD00.A2021034.0415.CORPUS_FULL_TEST.L1A_LAC --startnudge=0 --stopnudge=10
MODIS L1A version 6.0.6, built Aug 16 2019 12:46:26
modis_l1a: ERROR: MODIS L1A processing failed.
Please examine the LogStatus and LogUser files for more information.

attachment 1

attachment 2

attachment 3

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 4:36 pm America/New_York
by OB.DAAC - SeanBailey

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 4:41 pm America/New_York
by oo_processing
PS, I see it happening on my two installs? One in Tampa and and one in Saint Petersburg.
Here is the Log files from the Saint Petersburg failure (one is too big)
attachment 1

attachment 2

attachment 3

attachment 4

attachment 5

attachment 6

attachment 7

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 4:51 pm America/New_York
by OB.DAAC - SeanBailey
Well, if you "updated" and got the old leapsec and utcpole files...it doesn't matter where you run it, it will fail. 
Once we fix the distribution servers so they're serving out fresh soup, (and you update), the problem will go away.

Sean

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 5:21 pm America/New_York
by oo_processing
Sean,
Thanks. I hit the post button on this before seeing that you had responded to the first.
Again, my bad.

I have a cron that does this nightly, so it should be good almost when you are.

#!/bin/bash
#$1 should be satellite
source /shares/seadas-7.5.3/ocssw/OCSSW_bash.env
echo $OCSSWROOT
COMMAND="python $OCSSWROOT/scripts/update_luts.py $1 --verbose"
$COMMAND

All MODIS files erroring out like this (both real-time and reprocessing)

Posted: Wed Feb 03, 2021 5:51 pm America/New_York
by OB.DAAC - SeanBailey
BTW, looks like we're back :grin: