getanc places some data in non-standard folders

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
smasse
Posts: 25
Joined: Wed Dec 05, 2018 3:58 pm America/New_York
Answers: 0

getanc places some data in non-standard folders

by smasse » Thu Sep 22, 2022 9:06 pm America/New_York

Starting August 19, 2022 the scripts gatanc.py (SeaDAS 7.5) and getanc (SeaDAS V2022.0) started to store some ancillary
data in unconventionally named folders. The usual storage is .../ancillary/<year>/<day-of-year>. Note that the
behavior is the same with 7.5 and V2022.0 and that the processing is not affectedi. I have attached two examples
of getanc.py; before and after August 18.

On July 26, 2022 I ran the following getanc.py. The ancillary storage folder is normal:

/usr/local/seadas/7.5/run/scripts/getanc.py --verbose --refreshDB --no2 /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Searching database: /usr/local/seadas/7.5/var/ancillary_data.db
Determining pass start and end times...
Terra
()
Input file: /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Sensor : terra
Start time: 2022207044005
End time : 2022207044454
()
OBPG session started
Downloading 'N202220618_MET_NCEP_6h.hdf' to /users/seadas/MODIS/ancillary/2022/206
Found: /users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
Downloading 'N2022205_SST_OIV2AV_24h.nc' to /users/seadas/MODIS/ancillary/2022/205
*** WARNING: No optimal NO2 files found.
()
Created 'T20220726044000.gran.L1B_1KM.hdf.anc' l2gen parameter text file:

icefile=/users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
met1=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met2=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met3=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
sstfile=/users/seadas/MODIS/ancillary/2022/205/N2022205_SST_OIV2AV_24h.nc

()
*** WARNING: The following ancillary data types were missing or are not optimal: MET OZONE NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.
Substituting climatological data: /usr/local/seadas/7.5/share/common/ozone_climatology_v2014.hdf

On July 26, 2022 I ran the following getanc. The ancillary storage folder is different for some data:

/usr/local/seadas/2022.0/run/scripts/getanc --verbose --refreshDB --no2 /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
ancillary_data.db
Searching database: /usr/local/seadas/2022.0/var/log/ancillary_data.db
Determining pass start and end times...
Aqua

Input file: /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
Sensor : aqua
Start time: 2022-09-01T10:37:55
End time : 2022-09-01T10:44:48

OBPG session started
Downloading A20220901103723.gran.L1B_1KM.anc.server ( 0.00 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T090000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T090000.MET.NRT.nc ( 2.42 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T120000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T120000.MET.NRT.nc ( 2.41 MBs)
[==================================================]
...Done
Downloading '20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc' to /home/seadas/MODIS/ancillary/0220/901
Downloading 20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc ( 6.94 MBs)
[==================================================]
...Done
*** WARNING: No optimal NO2 files found.

Created 'A20220901103723.gran.L1B_1KM.hdf.anc' l2gen parameter text file:

icefile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
met1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
met2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
met3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
ozone2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
sstfile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc

*** WARNING: The following ancillary data types were missing or are not optimal: MET NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.

Tags:

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 644
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1
Been thanked: 1 time

Re: getanc places some data in non-standard folders

by OB SeaDAS - xuanyang02 » Fri Sep 23, 2022 10:19 am America/New_York

Thanks for reporting the issue. getanc.py(SeaDAS -7.5) and getanc(V2022.0) are returning a new set of ancillary files. The new filenames do not start with "NYYYY...", and getanc.py and getanc have trouble recognizing them. The bug was fixed. getanc will behave right in the next tag of OCSSW, but getanc.py will keep placing the ancillary files in the non-standard folders.

smasse
Posts: 25
Joined: Wed Dec 05, 2018 3:58 pm America/New_York
Answers: 0

Re: getanc places some data in non-standard folders

by smasse » Fri Sep 23, 2022 3:03 pm America/New_York

Thanks for the quick answer. I will modify the processing accordingly and wait for the next tag availability
P.S. Will a post be submitted in this forum when the next tag is available?

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 644
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1
Been thanked: 1 time

Re: getanc places some data in non-standard folders

by OB SeaDAS - xuanyang02 » Fri Sep 23, 2022 3:21 pm America/New_York

I will remember to reply to this post when V2022.1 comes out.

smasse
Posts: 25
Joined: Wed Dec 05, 2018 3:58 pm America/New_York
Answers: 0

Re: getanc places some data in non-standard folders

by smasse » Sat Sep 24, 2022 12:01 am America/New_York

Thank you kindly.

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 644
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1
Been thanked: 1 time

Re: getanc places some data in non-standard folders

by OB SeaDAS - xuanyang02 » Fri Nov 18, 2022 2:34 pm America/New_York

V2022.1 is available now.

Post Reply