New SST data in CMR?

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
earthengine_urs
Posts: 72
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 1 time

New SST data in CMR?

by earthengine_urs » Fri May 08, 2020 5:15 pm America/New_York

Hi,

I noticed that there is no https://oceandata.sci.gsfc.nasa.gov/ob/getfile/TERRA_MODIS.20200311.L3m.DAY.SST.sst.4km.nc (compare with the existing https://oceandata.sci.gsfc.nasa.gov/ob/getfile/TERRA_MODIS.20200310.L3m.DAY.SST.sst.4km.nc)

I'm trying to use CMR to list the available files, but can't figure out the concept id for these SST files - none of the entries mentioning Terra SST generate the right files: https://cmr.earthdata.nasa.gov/search/collections.xml?page_size=2000&provider=OB_DAAC

Help?

Thanks,
Simon

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 372
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

New SST data in CMR?

by OB ODPS - towens » Mon May 11, 2020 9:23 am America/New_York

Try this for the Concept ID: C1615934288-OB_DAAC
The missing SST files are being reprocessed now. They should be available via CMR search tomorrow.

Tommy

earthengine_urs
Posts: 72
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 1 time

New SST data in CMR?

by earthengine_urs » Mon May 11, 2020 3:43 pm America/New_York

Thank you, Tommy. Are the files ever supposed to be missing? My current code has a provision for handling missing files, and I'm wondering is this is still relevant.

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 372
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

New SST data in CMR?

by OB ODPS - towens » Mon May 11, 2020 4:11 pm America/New_York

There is a significant delay (15-30 day) between the availability of the NRT SST data and the Refined data, so data may be missing at the leading edge of the stream. Occasionally, A hiccup in the system may result in a missing file or two. Better to have the code prepared for missing data than not.

Tommy

earthengine_urs
Posts: 72
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 1 time

New SST data in CMR?

by earthengine_urs » Wed May 13, 2020 2:27 pm America/New_York

Tommy,

We see SST files for 20200310, but now there is no SST for 20200331. Worse, if I look at CMR, there are no files for any of the concepts we care about at all starting from Mar 15 (even though your site has the files):
        'C1458097478-OB_DAAC',
        'C1442922863-OB_DAAC',
        'C1442922871-OB_DAAC',
        'C1200034429-OB_DAAC',
        'C1615934288-OB_DAAC'

Eg, https://cmr.earthdata.nasa.gov/search/granules.json?concept_id%5b%5d=C1458097478-OB_DAAC&temporal%5b%5d=2020-03-15/2020-06-01&page_size=1000&page_num=1

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

New SST data in CMR?

by OB.DAAC - SeanBailey » Wed May 13, 2020 2:49 pm America/New_York

We have the standard "collections" and the NRT "collections" separated in CMR.  If you want near real-time data, you'll need to look for those concept_ids.  Or don't search by concept_id, but rather shortname/provider, e.g.:

https://cmr.earthdata.nasa.gov/search/granules.json?&page_size=200&sort_key=short_name&sort_key=start_date&short_name=MODIST_L3m_CHL_NRT&provider=OB_DAAC&temporal=2020-03-01,2020-05-13
https://cmr.earthdata.nasa.gov/search/granules.json?&page_size=200&sort_key=short_name&sort_key=start_date&short_name=MODIST_L3m_CHL&provider=OB_DAAC&temporal=2020-03-01,2020-05-13

That way you'll know which are NRT and which are not without having to cross reference any arbitrary ID defined by the CMR system when we added the "collection".

Sean

earthengine_urs
Posts: 72
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 1 time

New SST data in CMR?

by earthengine_urs » Wed May 13, 2020 3:05 pm America/New_York

Sean - so are the data starting from Mar 15 considered NRT? But there are SST files on https://oceandata.sci.gsfc.nasa.gov/MODIS-Terra/L3SMI/2020/091 that don't have "NRT" in their names. I'm confused...

There are no data starting on Mar 15 if I use the link in you format as well: https://cmr.earthdata.nasa.gov/search/granules.json?&page_size=200&sort_key=short_name&sort_key=start_date&short_name=MODIST_L3m_CHL&provider=OB_DAAC&temporal=2020-03-15,2020-05-13

I think I like using the concept ids slightly more (for consistency with other NASA datasets and because they are more narrow). Though for the oceancolor datasets I have to filter the search results anyway (eg, to only keep 4km files), so either approach is ok.

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 372
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

New SST data in CMR?

by OB ODPS - towens » Wed May 13, 2020 3:08 pm America/New_York

Because of the timing of the spacecraft orbit, the refined SST granules for March 31st require some data from April 1st, which will not be processed until the April refined ancillary data are released in mid-May.  Expect data for March 31 in the coming week.
The refined-ancillary versions of the Ocean Color data should be available out to April 25th. I'll look at EarthData search and see why they are not showing up.

Tommy

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

New SST data in CMR?

by OB.DAAC - SeanBailey » Wed May 13, 2020 3:17 pm America/New_York

...and OC NRT/refined does not equal SST NRT/refined.  There are different criteria for initiating the refined processing for the two. 

Sean

earthengine_urs
Posts: 72
Joined: Mon Jan 27, 2020 10:36 am America/New_York
Answers: 0
Has thanked: 3 times
Been thanked: 1 time

New SST data in CMR?

by earthengine_urs » Wed May 13, 2020 3:17 pm America/New_York

Thank you, Sean. To be clear, are the non-NRT SST files are expected to be added later than CHL etc? (I'm not quite sure what "refined-ancillary" means.)

Post Reply