ocssw/processing error getting ancillary data and Atmospheric correction

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
whu_lmq
Posts: 2
Joined: Thu Jun 30, 2022 5:40 am America/New_York

ocssw/processing error getting ancillary data and Atmospheric correction

by whu_lmq » Thu Jun 30, 2022 8:38 am America/New_York

When I use ocssw to download auxiliary data, the following statements will be output:

*** WARNING: The following ancillary data types were missing or are not optimal: MET
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.

I check the output directory and find that the auxiliary files have been downloaded successfully.But when I do data processing, an error occurs:

Ancillary data needed for processing, but either:
(1) does not exist or (2) does not contain ANC data associated with: V2018152175400.L1A_SNPP.nc
Program cannot continue

I don't know where the problem is and how to solve it.

Tags:

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 357
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by OB SeaDAS - xuanyang02 » Thu Jun 30, 2022 11:43 am America/New_York

We recently changed the return status of an api that getanc calls, which resulted in some inaccurate printouts regarding anc files, but you should get the anc files.

Running debug binaries...
/Users/bingyang/ocssw-V2021.2/bin
ancillary_data.db
VIIRS
icefile=/Users/*/ocssw-V2021.2/var/anc/2018/152/N2018152_SST_OIV2AV_24h.nc
met1=/Users/*/ocssw-V2021.2/var/anc/2018/152/N201815212_MET_NCEPR2_6h.hdf
met2=/Users/*/ocssw-V2021.2/var/anc/2018/152/N201815218_MET_NCEPR2_6h.hdf
met3=/Users/*/ocssw-V2021.2/var/anc/2018/153/N201815300_MET_NCEPR2_6h.hdf
ozone1=/Users/*/ocssw-V2021.2/var/anc/2018/152/N201815200_O3_AURAOMI_24h.hdf
ozone2=/Users/*/ocssw-V2021.2/var/anc/2018/153/N201815300_O3_AURAOMI_24h.hdf
ozone3=/Users/*/ocssw-V2021.2/var/anc/2018/153/N201815300_O3_AURAOMI_24h.hdf
sstfile=/Users/*/ocssw-V2021.2/var/anc/2018/152/N2018152_SST_OIV2AV_24h.nc


*** WARNING: The following ancillary data types were missing or are not optimal: MET
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.

You can just ignore the warnings for now.

Could you please check if you are getting the same anc files?

I am assuming your l2gen failed with the error you provided. Are you running l2gen on command line? What is your l2gen command? I was able to run l2gen on V2018152175400.L1A_SNPP.nc. Maybe you need to check your environment variable $OCSSWROOT?

Bing

sunyue2017
Posts: 3
Joined: Wed Apr 06, 2022 8:52 am America/New_York

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by sunyue2017 » Mon Jul 04, 2022 10:17 am America/New_York

Hi, Bing, thanks for your reply. I just met the same problem.

I used ocssw-T202202 to download anc files and process MODIS Aqua data - A2012001044500.L1A_LAC.bz2

all used *.anc *.atteph and ancillary.db have been downloaded.

2012-01-01T04:45:01.anc
2012-01-01T04:45:01.atteph
ancillary.db

However, when I checked the existence of anc files using following python code (l2gen has not been used yet)

conn = sqlite3.connect(anc_filepath)
cur = conn.cursor()
starttime = '2012-01-01T04:45:01'
result = cur.execute('select ' + 'attephstat' + ' from satfiles where starttime ="' + starttime + '"')
r = result.fetchone()

the same problem appeared..
Ancillary data needed for processing, but either:
(1) does not exist or (2) does not contain ATT/EPH data associated with: /scratch/sunyue/Sat_process/MODISA/CHS/2012/download/data_origin/A2012001044500.L1A_LAC.bz2

not sure whether the version of ocssw or some other factors caused this problem?

whu_lmq
Posts: 2
Joined: Thu Jun 30, 2022 5:40 am America/New_York

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by whu_lmq » Tue Jul 05, 2022 4:12 am America/New_York

Hello, when I use the auxiliary data of warning during download to process, there will be errors. However, when I use the auxiliary data downloaded before to process the corresponding data, the above error will not appear.

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 357
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by OB SeaDAS - xuanyang02 » Tue Jul 05, 2022 11:15 am America/New_York

sunyue2017,

There is no T202202 tag for OCSSW. Did you mean V2022.0? Can you run

Code: Select all

seadas_info
and post the result here?

Is the following block of code your own code or from our ocssw script? If it's from ocssw, which script? We need to debug and find out what caused the following print out --

"Ancillary data needed for processing, but either:
(1) does not exist or (2) does not contain ATT/EPH data associated with: /scratch/sunyue/Sat_process/MODISA/CHS/2012/download/data_origin/A2012001044500.L1A_LAC.bz2"

conn = sqlite3.connect(anc_filepath)
cur = conn.cursor()
starttime = '2012-01-01T04:45:01'
result = cur.execute('select ' + 'attephstat' + ' from satfiles where starttime ="' + starttime + '"')
r = result.fetchone()

Thanks,

Bing
Last edited by OB SeaDAS - xuanyang02 on Tue Jul 05, 2022 1:50 pm America/New_York, edited 4 times in total.

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 357
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by OB SeaDAS - xuanyang02 » Tue Jul 05, 2022 11:19 am America/New_York

whu_lmq,

You need to let us know more details before we can help you further.

Can you run

Code: Select all

seadas_info
and post the result there?

Thanks,

Bing

sunyue2017
Posts: 3
Joined: Wed Apr 06, 2022 8:52 am America/New_York

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by sunyue2017 » Wed Jul 06, 2022 12:01 am America/New_York

Dear Bing,

Thanks for your reply. I checked my code and found that some code worked for checking ancs files previously not working with now newly downloaded anc causing this error. The program can run normally now after I modified the corresponding code.

Thank again for your reply and suggestion.
Yue

sunyue2017
Posts: 3
Joined: Wed Apr 06, 2022 8:52 am America/New_York

Re: ocssw/processing error getting ancillary data and Atmospheric correction

by sunyue2017 » Wed Jul 06, 2022 12:02 am America/New_York

Dear Bing,

Thanks for your reply. I checked my code and found that some code worked for checking ancs files previously not working with now newly downloaded anc causing this error. The program can run normally now after I modified the corresponding code.

Thank again for your reply and suggestion.
Yue

Post Reply