HLS 2.0 granules for almost any December day are missing

[truf]

Hi,

I've fetched all granules since 2018 (in fact just their overiews) from the HLS2.0 dataset for a specific tile_id and found out that the amount of data dramatically differs from month to month. Especially for december.
Consider tile_id 39UUV and following commands:

wget https://github.com/nasa/HLS-Data-Resources/raw/main/bash/hls-bulk-download/getHLS.sh
chmod +x ./getHLS.sh
echo "39UUV" > /tmp/tiles

./getHLS.sh /tmp/tiles 2018-12-01 2018-12-31 /tmp/results
./getHLS.sh /tmp/tiles 2019-12-01 2019-12-31 /tmp/results
./getHLS.sh /tmp/tiles 2020-12-01 2020-12-31 /tmp/results
./getHLS.sh /tmp/tiles 2021-12-01 2021-12-31 /tmp/results
./getHLS.sh /tmp/tiles 2022-12-01 2022-12-31 /tmp/results
./getHLS.sh /tmp/tiles 2023-12-01 2023-12-31 /tmp/results
./getHLS.sh /tmp/tiles 2024-12-01 2024-12-31 /tmp/results

The output is:

granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
Downloading into /tmp/results/S30/2018/39/U/U/V/HLS.S30.T39UUV.2018335T080259.v2.0
Finished downloading into /tmp/results/S30/2018/39/U/U/V/HLS.S30.T39UUV.2018335T080259.v2.0
1 granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
Downloading into /tmp/results/S30/2019/39/U/U/V/HLS.S30.T39UUV.2019338T081311.v2.0
Finished downloading into /tmp/results/S30/2019/39/U/U/V/HLS.S30.T39UUV.2019338T081311.v2.0
1 granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
Downloading into /tmp/results/S30/2020/39/U/U/V/HLS.S30.T39UUV.2020338T081319.v2.0
Finished downloading into /tmp/results/S30/2020/39/U/U/V/HLS.S30.T39UUV.2020338T081319.v2.0
0 granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
1 granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
Downloading into /tmp/results/S30/2022/39/U/U/V/HLS.S30.T39UUV.2022337T081219.v2.0
Finished downloading into /tmp/results/S30/2022/39/U/U/V/HLS.S30.T39UUV.2022337T081219.v2.0
0 granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
0 granules to download
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value

So only 4 granules are available. 1 per 2018, 2019, 2020 and 2022.

There are at least 10 times more data for this tile_id for other months.
I guess this isn't the only tile that lacks december data.
Any ideas whats wrong with it?

[LP DAACx - dgolon]

Hi @truf Thanks for writing in. I'm having our developer of that script take a look to see what is going on. I'll report back when we have more information. Thanks --Danielle

[truf]

Hi!
Have you go any news re this problem? I don't sure it's a script issue, perhaps it's a db problem.

[mitchbon]

Is this a higher latitude area? Landsat and Sentinel-2 image collection decreases significantly during the winter (esp. December/January) in higher latitude areas. For example, above ~50N you will find very few (if any) images during those months (at least in Canada).

I assume this is due to lower sunlight conditions during this period.

[kristianbodolai]

Hi all, I'm having a very similar issue in a ~equator area, missing tiles from the STAC collection for december 2020. Code to reproduce it follows:

[code]
import pystac_client
hls_collections = ["HLSS30.v2.0"]
cmr_catalog = pystac_client.Client.open("https://cmr.earthdata.nasa.gov/stac/LPCLOUD/")

cmr_items = cmr_catalog.search(collections=hls_collections, bbox=[35, -1.5, 39,-0.5],datetime='2020-12-21').item_collection()
[/code]

This returns only 2 items, where a similar search in the copernicus dataspace returns 4.

[LP DAACx - dgolon]

Hello @kristianbodolai and @truf I apologize for the delay in response, but we have heard back from the science team. They have been able to recreate the issue and investigate why it has occurred. Below is a detailed explanation for why there are only 4 Granules for December and no L30 granules:

HLS processing discards granules of mean solar zenith angle >76° and of cloud cover >95%.

1. High Latitude
During high-latitude winter, the solar zenith angle is very large at the time of observation. Because atmospheric correction becomes less reliable at large solar zenith, a 76° threshold is used to eliminate such as Level-1 granules from processing.
For the tile 39UUV, there are 105 Sentinel-2 granules for the December of 2018-2023, but only 7 of them have a solar zenith angle < 76°. Of the remaining seven, three have butt cover >95%. As a result, only four meet the HLS processing criteria and four HLS S30 products were created, as the user reported. No L30 are available for December of 2018-2023 for similar reasons.
There can be apparent discrepancy in the HLS data regarding the 76° threshold. With the 76° mean solar zenith angle threshold, the metadata of some S30 and L30 show mean solar zenith angle slightly greater than 76°. This apparent discrepancy for S30 is caused by ESA’s derivation of solar zenith angle for the entire tile, no matter how much of the tile is actually observed, but HLS mean solar zenith metadata is only for the observed area. In the following example of a December granule over 39UUV, the mean solar zenith angle for the entire tile regardless of spatial coverage can be slightly less than 76°, based on ESA metadata, satisfying HLS processing criterion, but the angle for the observed portion of the tile can be slightly greater than 76° as the S30 metadata may show.
For Landsat, a WRS-2 scene used by HLS can have a mean solar zenith angle slightly <76°, but a portion of it gridded to an MGRS tile can have mean zenith angle slighter >76° for the MGRS tile.

2. Tropical Area
Persistent cloud cover is a problem for the tropical area. To save computing cost, HLS does not process granules with cloud cover >95%.
In the 2020-12-21 example, the bounding box [35 -1.5 39 -0.5] intercepts the following four granules, but the last two granules have cloud cover >95% and therefore are not processed by HLS.
S2B_MSIL1C_20201221T073229_N0209_R049_T37MDU_20201221T093009.SAFE
S2B_MSIL1C_20201221T073229_N0209_R049_T37MDV_20201221T093009.SAFE
S2B_MSIL1C_20201221T073229_N0209_R049_T37MEU_20201221T093009.SAFE 97.3108
S2B_MSIL1C_20201221T073229_N0209_R049_T37MEV_20201221T093009.SAFE 96.4579

Thank you,

HLS Science Team

[kristianbodolai]

Hello, thank you very much for your reply, this definitely solves the mystery!

I remember thinking about this as a potential cause back in the day, but I didn't see anything in the user guide, on a quick look again, I don't think it's there. Is there plans for updating it in the future? If so, it would be useful to include this kind of information.

[LP DAACx - dgolon]

Hi @kristianbodolai We'll speak with the team to see if they plan on updating the User Guide to reflect this information. In the meantime, we have a ticket to update the Known Issues sections on each of the HLS DOI Landing pages.
https://doi.org/10.5067/HLS/HLSS30.002
https://doi.org/10.5067/HLS/HLSL30.002

[kristianbodolai]

Thanks! it being in the known issues would be grand