Page 1 of 1

ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Thu Jun 30, 2022 8:38 am America/New_York
by whu_lmq
When I use ocssw to download auxiliary data, the following statements will be output:

*** WARNING: The following ancillary data types were missing or are not optimal: MET
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.

I check the output directory and find that the auxiliary files have been downloaded successfully.But when I do data processing, an error occurs:

Ancillary data needed for processing, but either:
(1) does not exist or (2) does not contain ANC data associated with: V2018152175400.L1A_SNPP.nc
Program cannot continue

I don't know where the problem is and how to solve it.

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Thu Jun 30, 2022 11:43 am America/New_York
by OB SeaDAS - xuanyang02
We recently changed the return status of an api that getanc calls, which resulted in some inaccurate printouts regarding anc files, but you should get the anc files.

Running debug binaries...
/Users/bingyang/ocssw-V2021.2/bin
ancillary_data.db
VIIRS
icefile=/Users/*/ocssw-V2021.2/var/anc/2018/152/N2018152_SST_OIV2AV_24h.nc
met1=/Users/*/ocssw-V2021.2/var/anc/2018/152/N201815212_MET_NCEPR2_6h.hdf
met2=/Users/*/ocssw-V2021.2/var/anc/2018/152/N201815218_MET_NCEPR2_6h.hdf
met3=/Users/*/ocssw-V2021.2/var/anc/2018/153/N201815300_MET_NCEPR2_6h.hdf
ozone1=/Users/*/ocssw-V2021.2/var/anc/2018/152/N201815200_O3_AURAOMI_24h.hdf
ozone2=/Users/*/ocssw-V2021.2/var/anc/2018/153/N201815300_O3_AURAOMI_24h.hdf
ozone3=/Users/*/ocssw-V2021.2/var/anc/2018/153/N201815300_O3_AURAOMI_24h.hdf
sstfile=/Users/*/ocssw-V2021.2/var/anc/2018/152/N2018152_SST_OIV2AV_24h.nc


*** WARNING: The following ancillary data types were missing or are not optimal: MET
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.

You can just ignore the warnings for now.

Could you please check if you are getting the same anc files?

I am assuming your l2gen failed with the error you provided. Are you running l2gen on command line? What is your l2gen command? I was able to run l2gen on V2018152175400.L1A_SNPP.nc. Maybe you need to check your environment variable $OCSSWROOT?

Bing

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Mon Jul 04, 2022 10:17 am America/New_York
by sunyue2017
Hi, Bing, thanks for your reply. I just met the same problem.

I used ocssw-T202202 to download anc files and process MODIS Aqua data - A2012001044500.L1A_LAC.bz2

all used *.anc *.atteph and ancillary.db have been downloaded.

2012-01-01T04:45:01.anc
2012-01-01T04:45:01.atteph
ancillary.db

However, when I checked the existence of anc files using following python code (l2gen has not been used yet)

conn = sqlite3.connect(anc_filepath)
cur = conn.cursor()
starttime = '2012-01-01T04:45:01'
result = cur.execute('select ' + 'attephstat' + ' from satfiles where starttime ="' + starttime + '"')
r = result.fetchone()

the same problem appeared..
Ancillary data needed for processing, but either:
(1) does not exist or (2) does not contain ATT/EPH data associated with: /scratch/sunyue/Sat_process/MODISA/CHS/2012/download/data_origin/A2012001044500.L1A_LAC.bz2

not sure whether the version of ocssw or some other factors caused this problem?

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Tue Jul 05, 2022 4:12 am America/New_York
by whu_lmq
Hello, when I use the auxiliary data of warning during download to process, there will be errors. However, when I use the auxiliary data downloaded before to process the corresponding data, the above error will not appear.

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Tue Jul 05, 2022 11:15 am America/New_York
by OB SeaDAS - xuanyang02
sunyue2017,

There is no T202202 tag for OCSSW. Did you mean V2022.0? Can you run

Code: Select all

seadas_info
and post the result here?

Is the following block of code your own code or from our ocssw script? If it's from ocssw, which script? We need to debug and find out what caused the following print out --

"Ancillary data needed for processing, but either:
(1) does not exist or (2) does not contain ATT/EPH data associated with: /scratch/sunyue/Sat_process/MODISA/CHS/2012/download/data_origin/A2012001044500.L1A_LAC.bz2"

conn = sqlite3.connect(anc_filepath)
cur = conn.cursor()
starttime = '2012-01-01T04:45:01'
result = cur.execute('select ' + 'attephstat' + ' from satfiles where starttime ="' + starttime + '"')
r = result.fetchone()

Thanks,

Bing

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Tue Jul 05, 2022 11:19 am America/New_York
by OB SeaDAS - xuanyang02
whu_lmq,

You need to let us know more details before we can help you further.

Can you run

Code: Select all

seadas_info
and post the result there?

Thanks,

Bing

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Wed Jul 06, 2022 12:01 am America/New_York
by sunyue2017
Dear Bing,

Thanks for your reply. I checked my code and found that some code worked for checking ancs files previously not working with now newly downloaded anc causing this error. The program can run normally now after I modified the corresponding code.

Thank again for your reply and suggestion.
Yue

Re: ocssw/processing error getting ancillary data and Atmospheric correction

Posted: Wed Jul 06, 2022 12:02 am America/New_York
by sunyue2017
Dear Bing,

Thanks for your reply. I checked my code and found that some code worked for checking ancs files previously not working with now newly downloaded anc causing this error. The program can run normally now after I modified the corresponding code.

Thank again for your reply and suggestion.
Yue