Page 1 of 1

Error in python Harmony submit call (that worked a couple weeks ago)

Posted: Wed Jun 28, 2023 12:07 pm America/New_York
by aronnem
Hello,

Recently I wrote a little prototype jupyter notebook that demonstrated using Harmony to subset L2 granules according to a spatial Lat/Lon box. The notebook is on github here, which includes the full output that I got from running it (probably on the commit day, or the day before - June 8 2023):
https://github.com/CLIMCAPS-tools/climcaps_cloud_prototypes/blob/main/harmony_spatial_subset_example.ipynb

When I try to run the exact same notebook, the harmony_client.submit(request) fails with this error:

---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[3], line 3
1 # submit and process request: I think this basically makes a bunch of subsetted nc4 files in a https served staging directory...
----> 3 job_id = harmony_client.submit(request)
4 harmony_client.wait_for_processing(job_id, show_progress=True)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/harmony/harmony.py:723, in Client.submit(self, request)
720 response = session.send(self._get_prepared_request(request))
722 if response.ok:
--> 723 job_id = (response.json())['jobID']
724 else:
725 self._handle_error_response(response)

KeyError: 'jobID'


From the python debugger, the JSON file clearly doesn't have a 'jobID' key. The abbreviated contents of the JSON file (I removed most of the 'links' for brevity) indicates some internal error that no operations can be performed on this dataset, even though I ran this exact request a few weeks ago with no error:

ipdb> response.json()
{'username': 'aronnem', 'status': 'successful', 'message': 'Returning direct download links because no operations can be performed on C1692982090-GES_DISC.', 'progress': 100, 'createdAt': '2023-06-28T15:53:37.823Z', 'updatedAt': '2023-06-28T15:53:37.823Z', 'dataExpiration': '2023-07-28T15:53:37.823Z', 'links': [{'href': 'https://data.gesdisc.earthdata.nasa.gov/data/JPSS1_Sounder_Level2/SNDRJ1IML2CCPRET.2/2021/213/SNDR.J1.CRIMSS.20210801T0048.m06.g009.L2_CLIMCAPS_RET.std.v02_53.G.211221121217.nc', 'title': 'G2187865862-GES_DISC', 'rel': 'data'}, ... < trimmed > ...], 'request': 'https://harmony.earthdata.nasa.gov/SNDRJ1IML2CCPRET/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?forceAsync=true&subset=lat(50%3A55)&subset=lon(10%3A15)&subset=time(%222021-08-01T00%3A00%3A00%22%3A%222021-08-02T00%3A00%3A00%22)', 'numInputGranules': 5}


Any ideas? Did expected syntax of the request change recently?

Thank you!

Re: Error in python Harmony submit call (that worked a couple weeks ago)

Posted: Thu Jul 13, 2023 6:54 pm America/New_York
by GES DISC - mgreene
Hello,
Could you please try your request again, and if you receive an error you may want to open up an issue in the Harmony Github repo: https://github.com/nasa/harmony

Hope this helps.