getanc places some data in non-standard folders
Posted: Thu Sep 22, 2022 9:06 pm America/New_York
Starting August 19, 2022 the scripts gatanc.py (SeaDAS 7.5) and getanc (SeaDAS V2022.0) started to store some ancillary
data in unconventionally named folders. The usual storage is .../ancillary/<year>/<day-of-year>. Note that the
behavior is the same with 7.5 and V2022.0 and that the processing is not affectedi. I have attached two examples
of getanc.py; before and after August 18.
On July 26, 2022 I ran the following getanc.py. The ancillary storage folder is normal:
/usr/local/seadas/7.5/run/scripts/getanc.py --verbose --refreshDB --no2 /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Searching database: /usr/local/seadas/7.5/var/ancillary_data.db
Determining pass start and end times...
Terra
()
Input file: /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Sensor : terra
Start time: 2022207044005
End time : 2022207044454
()
OBPG session started
Downloading 'N202220618_MET_NCEP_6h.hdf' to /users/seadas/MODIS/ancillary/2022/206
Found: /users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
Downloading 'N2022205_SST_OIV2AV_24h.nc' to /users/seadas/MODIS/ancillary/2022/205
*** WARNING: No optimal NO2 files found.
()
Created 'T20220726044000.gran.L1B_1KM.hdf.anc' l2gen parameter text file:
icefile=/users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
met1=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met2=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met3=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
sstfile=/users/seadas/MODIS/ancillary/2022/205/N2022205_SST_OIV2AV_24h.nc
()
*** WARNING: The following ancillary data types were missing or are not optimal: MET OZONE NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.
Substituting climatological data: /usr/local/seadas/7.5/share/common/ozone_climatology_v2014.hdf
On July 26, 2022 I ran the following getanc. The ancillary storage folder is different for some data:
/usr/local/seadas/2022.0/run/scripts/getanc --verbose --refreshDB --no2 /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
ancillary_data.db
Searching database: /usr/local/seadas/2022.0/var/log/ancillary_data.db
Determining pass start and end times...
Aqua
Input file: /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
Sensor : aqua
Start time: 2022-09-01T10:37:55
End time : 2022-09-01T10:44:48
OBPG session started
Downloading A20220901103723.gran.L1B_1KM.anc.server ( 0.00 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T090000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T090000.MET.NRT.nc ( 2.42 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T120000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T120000.MET.NRT.nc ( 2.41 MBs)
[==================================================]
...Done
Downloading '20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc' to /home/seadas/MODIS/ancillary/0220/901
Downloading 20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc ( 6.94 MBs)
[==================================================]
...Done
*** WARNING: No optimal NO2 files found.
Created 'A20220901103723.gran.L1B_1KM.hdf.anc' l2gen parameter text file:
icefile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
met1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
met2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
met3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
ozone2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
sstfile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
*** WARNING: The following ancillary data types were missing or are not optimal: MET NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.
data in unconventionally named folders. The usual storage is .../ancillary/<year>/<day-of-year>. Note that the
behavior is the same with 7.5 and V2022.0 and that the processing is not affectedi. I have attached two examples
of getanc.py; before and after August 18.
On July 26, 2022 I ran the following getanc.py. The ancillary storage folder is normal:
/usr/local/seadas/7.5/run/scripts/getanc.py --verbose --refreshDB --no2 /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Searching database: /usr/local/seadas/7.5/var/ancillary_data.db
Determining pass start and end times...
Terra
()
Input file: /users/seadas/data/20220726044000.eam1/T20220726044000.gran.L1B_1KM.hdf
Sensor : terra
Start time: 2022207044005
End time : 2022207044454
()
OBPG session started
Downloading 'N202220618_MET_NCEP_6h.hdf' to /users/seadas/MODIS/ancillary/2022/206
Found: /users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
Downloading 'N2022205_SST_OIV2AV_24h.nc' to /users/seadas/MODIS/ancillary/2022/205
*** WARNING: No optimal NO2 files found.
()
Created 'T20220726044000.gran.L1B_1KM.hdf.anc' l2gen parameter text file:
icefile=/users/seadas/MODIS/ancillary/2022/205/N202220500_SEAICE_NSIDC_24h.hdf
met1=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met2=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
met3=/users/seadas/MODIS/ancillary/2022/206/N202220618_MET_NCEP_6h.hdf
sstfile=/users/seadas/MODIS/ancillary/2022/205/N2022205_SST_OIV2AV_24h.nc
()
*** WARNING: The following ancillary data types were missing or are not optimal: MET OZONE NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.
Substituting climatological data: /usr/local/seadas/7.5/share/common/ozone_climatology_v2014.hdf
On July 26, 2022 I ran the following getanc. The ancillary storage folder is different for some data:
/usr/local/seadas/2022.0/run/scripts/getanc --verbose --refreshDB --no2 /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
ancillary_data.db
Searching database: /usr/local/seadas/2022.0/var/log/ancillary_data.db
Determining pass start and end times...
Aqua
Input file: /home/seadas/data/20220901103723.epm1/A20220901103723.gran.L1B_1KM.hdf
Sensor : aqua
Start time: 2022-09-01T10:37:55
End time : 2022-09-01T10:44:48
OBPG session started
Downloading A20220901103723.gran.L1B_1KM.anc.server ( 0.00 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T090000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T090000.MET.NRT.nc ( 2.42 MBs)
[==================================================]
...Done
Downloading 'GMAO_FP.20220901T120000.MET.NRT.nc' to /home/seadas/MODIS/ancillary/MAO_/FP.
Downloading GMAO_FP.20220901T120000.MET.NRT.nc ( 2.41 MBs)
[==================================================]
...Done
Downloading '20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc' to /home/seadas/MODIS/ancillary/0220/901
Downloading 20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc ( 6.94 MBs)
[==================================================]
...Done
*** WARNING: No optimal NO2 files found.
Created 'A20220901103723.gran.L1B_1KM.hdf.anc' l2gen parameter text file:
icefile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
met1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
met2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
met3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone1=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T090000.MET.NRT.nc
ozone2=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
ozone3=/home/seadas/MODIS/ancillary/MAO_/FP./GMAO_FP.20220901T120000.MET.NRT.nc
sstfile=/home/seadas/MODIS/ancillary/0220/901/20220901120000-CMC-L4_GHRSST-SSTfnd-CMC0.1deg-GLOB-v02.0-fv03.0.nc
*** WARNING: The following ancillary data types were missing or are not optimal: MET NO2
*** Beware that certain MET and OZONE files just chosen by this program are not optimal.
*** For near real-time processing the remaining files may become available soon.