Page 1 of 1

getanc places some data in non-standard folders

Posted: Thu Sep 22, 2022 9:06 pm America/New_York
by smasse
Starting August 19, 2022 the scripts gatanc.py (SeaDAS 7.5) and getanc (SeaDAS V2022.0) started to store some ancillary
data in unconventionally named folders. The usual storage is .../ancillary/<year>/<day-of-year>. Note that the
behavior is the same with 7.5 and V2022.0 and that the processing is not affectedi. I have attached two examples
of getanc.py; before and after August 18.

On July 26, 2022 I ran the following getanc.py. The ancillary storage folder is normal:

/usr/local/seadas/7.5/run/scripts/getanc.py --verbose --refreshDB --no2 /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Searching database: /usr/local/seadas/7.5/var/ancillary_data.db
Determining pass start and end times...
Terra
()
Input file: /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Sensor : terra
Start time: 2022207044005
End time : 2022207044454
()
OBPG session started
Downloading 'N202220618_MET_NCEP_6h.hdf' to /users/seadas/MODIS/ancillary/2022/206
Found: /users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
Downloading 'N2022205_SST_OIV2AV_24h.nc' to /users/seadas/MODIS/ancillary/2022/205
*** WARNING: No optimal NO2 files found.
()
Created 'T20220726044000.gran.L1B_1KM.hdf.anc' l2gen parameter text file:

icefile=/users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
met1=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met2=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met3=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
sstfile=/users/seadas/MODIS/ancillary/2022/205/N2022205_SST_OIV2AV_24h.nc

()
*** WARNING: The following ancillary data types were missing or are not optimal: MET OZONE NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.
Substituting climatological data: /usr/local/seadas/7.5/share/common/ozone_climatology_v2014.hdf

On July 26, 2022 I ran the following getanc. The ancillary storage folder is different for some data:

/usr/local/seadas/2022.0/run/scripts/getanc --verbose --refreshDB --no2 /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
ancillary_data.db
Searching database: /usr/local/seadas/2022.0/var/log/ancillary_data.db
Determining pass start and end times...
Aqua

Input file: /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
Sensor : aqua
Start time: 2022-09-01T10:37:55
End time : 2022-09-01T10:44:48

OBPG session started
Downloading A20220901103723.gran.L1B_1KM.anc.server ( 0.00 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T090000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T090000.MET.NRT.nc ( 2.42 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T120000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T120000.MET.NRT.nc ( 2.41 MBs)
[==================================================]
...Done
Downloading '20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc' to /home/seadas/MODIS/ancillary/0220/901
Downloading 20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc ( 6.94 MBs)
[==================================================]
...Done
*** WARNING: No optimal NO2 files found.

Created 'A20220901103723.gran.L1B_1KM.hdf.anc' l2gen parameter text file:

icefile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
met1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
met2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
met3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
ozone2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
sstfile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc

*** WARNING: The following ancillary data types were missing or are not optimal: MET NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.

Re: getanc places some data in non-standard folders

Posted: Fri Sep 23, 2022 10:19 am America/New_York
by OB SeaDAS - xuanyang02
Thanks for reporting the issue. getanc.py(SeaDAS -7.5) and getanc(V2022.0) are returning a new set of ancillary files. The new filenames do not start with "NYYYY...", and getanc.py and getanc have trouble recognizing them. The bug was fixed. getanc will behave right in the next tag of OCSSW, but getanc.py will keep placing the ancillary files in the non-standard folders.

Re: getanc places some data in non-standard folders

Posted: Fri Sep 23, 2022 3:03 pm America/New_York
by smasse
Thanks for the quick answer. I will modify the processing accordingly and wait for the next tag availability
P.S. Will a post be submitted in this forum when the next tag is available?

Re: getanc places some data in non-standard folders

Posted: Fri Sep 23, 2022 3:21 pm America/New_York
by OB SeaDAS - xuanyang02
I will remember to reply to this post when V2022.1 comes out.

Re: getanc places some data in non-standard folders

Posted: Sat Sep 24, 2022 12:01 am America/New_York
by smasse
Thank you kindly.

Re: getanc places some data in non-standard folders

Posted: Fri Nov 18, 2022 2:34 pm America/New_York
by OB SeaDAS - xuanyang02
V2022.1 is available now.