Missing HLS Granules between 2024-03-05 & 2024-03-12

[matthew_cox_omnia]

It appears that there may be a number of missing granules between 3/5 and 3/12. When I do a search for granules between those dates: https://search.earthdata.nasa.gov/search/granules?p=C2021957657-LPCLOUD&pg%5B0%5D%5Bv%5D=f&pg%5B0%5D%5Bgsk%5D=-start_date&qt=2024-03-05T00%3A00%3A00.000Z%2C2024-03-12T23%3A59%3A59.999Z&tl=1710422347!3!!&lat=27.56229001520878&long=-144.9140625 1184 results are returned, which is just 3% of the results from an 8 day window just before that: https://search.earthdata.nasa.gov/search/granules?p=C2021957657-LPCLOUD&pg%5B0%5D%5Bv%5D=f&pg%5B0%5D%5Bgsk%5D=-start_date&qt=2024-02-26T00%3A00%3A00.000Z%2C2024-03-04T23%3A59%3A59.999Z&tl=1710422347!3!!&lat=27.56229001520878&long=-144.9140625.

The effect is even more pronounced comparing 125 granules returned between 3/5-3/9, which is < 1% of the 20K returned the 5 day window 8 days prior:
https://search.earthdata.nasa.gov/search/granules?p=C2021957657-LPCLOUD&pg%5B0%5D%5Bv%5D=f&pg%5B0%5D%5Bgsk%5D=-start_date&qt=2024-03-05T00%3A00%3A00.000Z%2C2024-03-09T23%3A59%3A59.999Z&tl=1710422347!3!!&lat=27.56229001520878&long=-144.9140625 vs https://search.earthdata.nasa.gov/search/granules?p=C2021957657-LPCLOUD&pg%5B0%5D%5Bv%5D=f&pg%5B0%5D%5Bgsk%5D=-start_date&qt=2024-02-26T00%3A00%3A00.000Z%2C2024-03-01T23%3A59%3A59.999Z&tl=1710422347!3!!&lat=27.56229001520878&long=-144.9140625 .

Any ideas why this occurred and whether those missing granules will be posted eventually? I suspect it's related to this topic:
viewtopic.php?t=5307#p18400

[LP DAAC - dgolon]

Hi @matthew_cox_omnia Thanks for bringing this to our attention. We'll check with our SDLC team and will report back soon. -- Danielle

[LP DAAC - lien]

Hi @matthew_cox_omnia
Sorry for the delay. It appears now processing has caught up and closed the data gaps. Thanks for your patience.
Brett

[matthew_cox_omnia]

Hi Brett and Danielle,

Thank you both for your prompt responses and assistance in resolving the issue with the missing HLS granules. I'm can see that the data gaps have now been filled, and I appreciate the efforts of the SDLC team in catching up with the processing.

This incident has highlighted the importance of continuous monitoring and timely data availability for our work. Can you share any insights on how often these kinds of gaps might occur and what measures are in place to detect and address them promptly? It would be great to understand the system's reliability and any steps we, as users, can take to mitigate such issues in the future.

Thanks again for your support and for keeping the community informed.
Matthew

[LP DAAC - lien]

Hi Matthew,

Please see below from a data manager:
Infrequent and unexpected data gaps can be caused by a number of issues, including delays in being able to access the input and/or ancillary data needed to generate the products. Since the provider, in this case HLS, is getting its input data upstream (Sentinel and Landsat), any delays or production issues in those data streams will directly affect HLS processing. The LP DAAC receives reports containing a list of the files sent to us by the SDS and placed on their https server or in their s3 bucket. The frequency of these reports may vary by provider. HLS sends them daily. Other providers may choose a different schedule. These reports will list the granule files that have been sent to the LP DAAC since the last report was generated. The reports will contain information including collection short name, collection version ID, granule id, granule filename, file size, timestamp and (optional) checksum value.

Although these data gaps are quite infrequent, in the case that they persist over a larger period of time, the provider generally alerts the LP DAAC and we are able to alert user’s via news announcements. In this case, we reached out to the HLS team and they acknowledged the gap and that it would be filled in momentarily.

Thanks,
Brett

[matthew_cox_omnia]

Hi Brett,

Thank you for the detailed explanation and insights from the data manager regarding the data gaps and the processes involved in monitoring and reporting these issues. It's clear that the LP DAAC is reliant on the upstream providers and their reporting mechanisms to identify and address these gaps. Your transparency in sharing how these processes work is greatly appreciated.

Given the potential impact of these data gaps on our operations, we are considering developing our own monitoring mechanisms to detect any anomalies or gaps in the data more proactively. This should help us to be more resilient to such occurrences and ensure the continuity of our work.

Could you advise if the forum is the preferred channel for us to report any unusual data issues we might encounter, or if there's a more direct communication channel that would be more appropriate for such alerts? We want to ensure that any concerns we have are communicated effectively and in a manner that is most useful for your team.

Thanks again for your support and looking forward to your guidance on the best way to proceed with reporting.

Best regards,
Matthew

[LP DAAC - dgolon]

Hi @matthew_cox_omnia Our team checks the forum on a regular basis Monday through Friday, and we receive a notification anytime you use the "lpdaac" tag, so you are free to continue to post here if you would like. It will also allow other users to chime in on a post, if they are seeing the same issue. If you would prefer, you can always email lpdaac@usgs.gov instead and that will create a ticket in our ticketing system and will alert us as soon as we receive your email. Either option works for us. Some users prefer to communicate through the forum, and some prefer to email in. Please feel free to use whichever is easier for you. If you do prefer to post to the forum, please make sure to use the tag feature so your post will be easy for us to find. Thanks --Danielle