Subsetting MUR-JPL-L4-GLOB-v4.1 data?

[negri]

Thanks for the last information. It worked.
but know, I have another issue.

In order to contextualize, I'm following this post (https://github.com/podaac/tutorials/blob/master/notebooks/opendap/MUR-OPeNDAP.ipynb)

1) my request is OK --> manager.add_password(None, endpoint, username, password)
r = requests.get('https://cmr.earthdata.nasa.gov/search/granules.umm_json?collection_concept_id=C1996881146-POCLOUD&temporal=2019-01-01T10:00:00Z,2019-01-05T00:00:00Z&pageSize=365')
r.status_code
Out[140]: <Response [200]>

2) Next, I got the opendap product...
'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/GHRSST%20Level%204%20MUR%20Global%20Foundation%20Sea%20Surface%20Temperature%20Analysis%20(v4.1)/granules/20190101090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1'

3) and... I added the extension '.dap.nc4?dap4'
'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/GHRSST%20Level%204%20MUR%20Global%20Foundation%20Sea%20Surface%20Temperature%20Analysis%20(v4.1)/granules/20190101090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.dap.nc4?dap4'

4) next...
required_variables = {'analysed_sst[0:1:0][000:1:9000][000:1:9000]',
'analysis_error[0:1:0][000:1:9000][000:1:9000]',
'lat[000:1:9000]',
'lon[000:1:9000]',
'time[0:1:0]'}

5) and finally (here I have the issue)

basename = os.path.basename(data_url)
request_params = {'dap4.ce': ';'.join(required_variables)}
response = requests.get(data_url, params=request_params, headers={'Accept-Encoding': 'identity'})

response
<Response [401]>
response.text
Out[154]: 'HTTP Basic: Access denied.\n'

Please, some sugestion ?

[PODAAC - jmcnelis]

Yes, First stage I used the following command
pip install podaac-data-subscriber. But it doest work to crop a domain.

Then I used
pip install podaac-data-subscriber==1.15.0a3

and trying to crop data as follows:
podaac-data-downloader -c MUR-JPL-L4-GLOB-v4.1 -d ./data --start-date 2003-01-01T09:00:00Z --end-date 2022-12-31T09:00:00Z
-b="135,-50,175,-10" --subset

Still, the global data is downloading but not getting the cropping domain as specified.

Thanks for your further instructions.

Hi @ran252_unsw,

You need to give the --subset flag before the -b option, like this:

"""
podaac-data-downloader -c MUR-JPL-L4-GLOB-v4.1 -d ./data --start-date 2003-01-01T09:00:00Z --end-date 2022-12-31T09:00:00Z --subset -b="135,-50,175,-10"
"""

I recommend setting smaller timeframes for your subset orders, and downloading them in batches. The podaac-data-downloader will need to wait for an enormous number of subsetting operations to complete when requesting 20 years of data.

Jack

[xcancel]

Apparently PODAAC Downloader through Harmony subsetting is broken for older GHRSST MUR data. I also had problems when trying to run the mentioned script for downloading 2002/2003 MUR files. The example provided by Wenhao Li using files from 2022 works fine, but if we try to apply it for 2002 files (for example), Harmony subsetting brokes and we receive an "Error processing harmony subsetting request".

Initially I thought it was a problem caused by trying to download multiple files at once, but even subsetting one by one is impossible. I recommend the PODAAC team to check why this is happening.

Edit: I just tried to subset download through wget using OPeNDAP granules and is working fine. Not sure why PODAAC Downloader can't access these older GHRSST MUR data.

Thank you very much.

[PODAAC - wenhaoli]

Thank you for finding this issue. We will investigate and let you know once we have the answer.

[PODAAC - wenhaoli]

Hi,

PO.DAAC team has discovered the issue, which is related with old OPeNDAP dmrpp files. Since late of 2023, the most recent MUR granules has been updated with the OPeNDAP dmrpp version 3.21.0-40, which has resolved this issue. So, we have to update dmrpp files for all MUR granules. Most likely, it will take about two week for the updating. Sorry for the inconvenience.

Thanks,

Wenhao

[PODAAC - celiaoued]

Hello,

OPeNDAP should work now for the older MUR files. Before trying again, you should manually delete a hidden .harmony file in the download folder you were using for the Data Subscriber command:

rm ./data/.harmony

[xcancel]

Dear PODAAC Team,

Thank you very much for your support. The PODAAC Downloader seems to be working fine for older GHRSST MUR files now.
However, can you please check the following granules?

20210220090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1 (2021-02-20)
20210221090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1 (2021-02-21)
20221109090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1 (2022-11-09)
20230317090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1 (2023-03-17)

I've been trying to download them, but both wget and PODAAC Downloader method returns me a 0 bytes file or no file at all.

Best,
Gabriel

[PODAAC - edward.m.armstrong]

Hi Gabriel,

I did confirm we have missing granules for Feb 20-21, 2021. We will try to recreate those. Investigating the other dates.

regards,
Ed

[PODAAC - edward.m.armstrong]

Also this granule from 2023 has no OPeNDAP link which is the service used for subsetting, so it will not work with the downloader. Investigating.

https://cmr.earthdata.nasa.gov/virtual-directory/collections/C1996881146-POCLOUD/temporal/2023/03/16