Converting GEDI h5 to GeoJSON

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
erynchang
Posts: 2
Joined: Sun Jan 23, 2022 8:55 pm America/New_York
Answers: 0

Converting GEDI h5 to GeoJSON

by erynchang » Sun Jan 23, 2022 11:34 pm America/New_York

Dear GEDI team,

I am a PhD student working in Australia and would like to use GEDI data as environmental covariates in my occupancy models for long-term monitoring of specific aboreal mammal. I have downloaded the GEDI L2B version 2 for my region using Earthdata search following this video tutorial (https://lpdaac.usgs.gov/resources/e-lea ... n-studies/). The data was successfully downloaded (see attached screenshot earthdatasearch.jpg).

And then I would like to convert the granules to GeoJSON files so that I can retrieve the attribute tables. I followed this tutorial and python script (https://git.earthdata.nasa.gov/projects ... ter/browse), but at keep getting this error when executing the script (see attached screenshot OutOfIndexError.jpg).

Processing file: processed_GEDI02_B_2019108185228_O01971_01_T00922_02_003_01_V002.h5 (1/160)
Traceback (most recent call last):
File "GEDI_Subsetter.py", line 157, in <module>
lat = [l for l in beamSDS if sdsSubset[0] in l][0]
IndexError: list index out of range

I tried with both GeoJSON and bounding box as an argument of --roi but neither of them works. Could you please advise me where goes wrong?

Thank you in advance!

---------

Attached also the downloaded folder structure and the h5 files in the input directory (GEDI_files.png).
The codes I am using in the command prompt are as below:

conda create -n gedi -c conda-forge --yes python=3.7.9 h5py shapely geopandas pandas

cd C:\Users\jc745612\Desktop\GEDI\gedi-subsetter

activate gedi

python GEDI_Subsetter.py --dir "C:\Users\jc745612\Desktop\GEDI\gediwt_2" --roi map.geojson

python GEDI_Subsetter.py --dir "C:\Users\jc745612\Desktop\GEDI\gediwt_2" --roi '-16.8,144.1,-19.7,147.8'
Attachments
GEDI files
GEDI files
GEDI_files.png (339.91 KiB) Not viewed yet
Python executing error
Python executing error
OutOfIndexError.jpg (224.6 KiB) Not viewed yet
Earthdata Search GEDI L2B v2
Earthdata Search GEDI L2B v2
earthdatasearch.jpg (95.4 KiB) Not viewed yet

Tags:

LP DAAC - jwilson
User Services
User Services
Posts: 266
Joined: Mon Sep 30, 2019 12:39 pm America/New_York
Answers: 1
Has thanked: 9 times

Re: Converting GEDI h5 to GeoJSON

by LP DAAC - jwilson » Thu Jan 27, 2022 11:33 am America/New_York

We have forwarded your inquiry to our Subject Matter Expert for further assistance.

LP DAAC - afriesz
Subject Matter Expert
Subject Matter Expert
Posts: 59
Joined: Tue Nov 12, 2019 4:02 pm America/New_York
Answers: 2
Been thanked: 2 times

Re: Converting GEDI h5 to GeoJSON

by LP DAAC - afriesz » Wed Feb 02, 2022 9:13 am America/New_York

@erynchang ,

I haven’t been able to replicate the issue you are experiencing. Here is what I’ve tried.

First, I cloned the repository (https://git.earthdata.nasa.gov/projects ... ter/browse) to a local directory, and I ran the Python Environment setup instructions (i.e., conda create -n gedi -c conda-forge --yes python=3.7 h5py shapely geopandas pandas).

Next, I replicated you’re geojson file using the coordinates from your previous post (i.e., '-16.8,144.1,-19.7,147.8').
I used the same coordinates to identify GEDI L2B granules in Earthdata Search. I downloaded two GEDI granule is two ways. The first way was to download the granules as-is, with no spatial subsetting. The second was to download the files that had been spatially subset using Earthdata Search’s Customize > Spatial Subset option.

I also found the granule you mentioned in your post (i.e., processed_GEDI02_B_2019108185228_O01971_01_T00922_02_003_01_V002.h5) in Earthdata Search and, again, downloaded the entire granule and a spatially subset version of the data.

I ran the GEDI_Subsetter.py script against all the files and experienced no fatal error. For each file a valid geojson file was created. I tried both --roi options, specifying both the geojson file I created ( --roi map.geojson) and the corner coordinates ( --roi '-16.8,144.1,-19.7,147.8'), which both ran successfully.

Unless you have any additional details I didn’t’ consider, I recommend re-downloading/cloning the repo (https://git.earthdata.nasa.gov/projects ... ter/browse) and creating a new Python environment to execute the code. If your issue persists, please contact the LP DAAC User Services (LPDAAC@usgs.gov) and we can set up a time to discuss/troubleshoot.

erynchang
Posts: 2
Joined: Sun Jan 23, 2022 8:55 pm America/New_York
Answers: 0

Re: Converting GEDI h5 to GeoJSON

by erynchang » Thu Feb 24, 2022 9:01 am America/New_York

Hi Afriesz,

Thank you so much for testing the data. Indeed I subset the layers on earthdata search and that's probably why the script failed. I re-downloaded the whole set without subsetting and it worked!

fionascarff
Posts: 2
Joined: Fri May 12, 2023 11:42 pm America/New_York
Answers: 0

Re: Converting GEDI h5 to GeoJSON

by fionascarff » Sat May 13, 2023 12:06 am America/New_York

Hello folks,

I am a postdoc in Australia, I would like to use the GEDI products as covariates for species distribution models for cockatoos. I have a problem that sems very similar to this one posted by Eryn Chang above. I have successfully downloaded some granules using NASA's Earthdata search function [https://search.earthdata.nasa.gov/search], and need only to convert them to GeoJSON. I have followed this tutorial to set up the Python environment and try to run the GEDI Subsetter [https://git.earthdata.nasa.gov/projects/LPDUR/repos/gedi-subsetter/browse] . I set it up in the Terminal interface of my M1 Mac like this:

(base) Fionas-iMac:~ tonynfi$ conda create -n gedi -c conda-forge --yes python=3.9 h5py shapely geopandas pandas

#navigate to directory where GEDI_Subsetter.py has been downloaded to:
(base) Fionas-iMac:~ tonynfi$ cd Documents/Fiona/Cockatoos/Cockatoos-2023/data/GEDI-for-processing

#activate GEDI:
(base) Fionas-iMac:GEDI-for-processing tonynfi$ source activate gedi

#Convert granule to GeoJSON:
(gedi) Fionas-iMac:GEDI-for-processing tonynfi$ python GEDI_Subsetter.py --dir 59205/2433458444 --roi ext59140.shp

However, I get this error message:
geometry
0 POLYGON ((116.25000 -32.50000, 116.25000 -32.0...
Processing file: processed_GEDI02_B_2019123141024_O02201_01_T00173_02_003_01_V002.h5 (1/1)
Traceback (most recent call last):
File "/Users/tonynfi/Documents/Fiona/Cockatoos/Cockatoos-2023/data/GEDI-for-processing/GEDI_Subsetter.py", line 159, in <module>
lat = [l for l in beamSDS if sdsSubset[0] in l][0]
IndexError: list index out of range

I have tried rerunning the installation code as suggested, but it returns the same error. I have also tried something that was successful for Eryn, which was to download the granules from Earthdata without subsetting the layers, but I still get the same error. Within Earthdata Search, I used temporal subsetting (1 Jan 2018 to 31 Dec 2019) and spatial subsetting (SW corner-32.5,116, NE corner -32,117) to identify relevant granules, and then variously tried downloading those granules
a) using the Customize function with the same spatial and temporal subsets
b) same as (a) but also with layer subsetting (layer rh100 only)
c) using the Direct Download option without any subsetting.

Altogether I would like to obtain data on a region spanning a degree range of 112E to 129E and 35.5S to 24.5S, on something like a 250m grid. I only need a snapshot, not a time series, but have set the temporal subsetting to span two years to allow the satellite to cover the area at that ~250m resolution, not sure if I that is too long a period and that could be contributing to the problem?

Any help would be much appreciated, thanks very much in advance for your time!

fionascarff
Posts: 2
Joined: Fri May 12, 2023 11:42 pm America/New_York
Answers: 0

Re: Converting GEDI h5 to GeoJSON

by fionascarff » Wed May 24, 2023 12:06 am America/New_York

Hello folks,

Just a follow up on that one, with a solution. I expect it is quite hard to provide help on this matter, because the problem isn't able to be replicated at the Earthdata end. I have switched to using the Google Earth Engine Code Editor to run the query. I was not sure whether the problem I was encountering might reflect a memory constraint in my local network, so this was a means of eliminating that as a limitation. The GEE Code Editor also enables raster exports in Geotiff format, so that avoids the difficulty of converting from hdf5 format. There's lots of useful help on the GEE code editor here: https://developers.google.com/earth-engine/guides/playground
Thanks to everyone at Earthdata for the very useful resources you provide.

Best, Fiona

milansamanta
Posts: 1
Joined: Fri Nov 24, 2023 3:12 am America/New_York
Answers: 0

Re: Converting GEDI h5 to GeoJSON

by milansamanta » Sat Nov 25, 2023 6:29 am America/New_York

Respected Sir/Madam, I have follow this link for my work https://git.earthdata.nasa.gov/projects/LPDUR/repos/gedi-subsetter/browse
During running python code for GEDI02_A data processing for converting GEDI h5 to GeoJSON file, their in the python code environment one error is showing day by day again and again, though i installed Python version 3.9, and install Anaconda, and gedi-subsetter.py file and downloded GEDI02_A file ("processed_GEDI02_A_2023047075930_O23673_03_T02630_02_003_02_V002.h5"), I put all these are in the same dir (--dir) in my window or same folder. during code ruining "(python GEDI_Subsetter.py --dir D:\10_12_2022\gedi\gedi-subsetteri --roi StudyArea.geojson") their output folder created but in the output folder their is empty or not have any file or GeoJSON file in this output folder. And in the python code their showing error code (AttributeError: 'DataFrame' object has no attribute 'append'). I attached this error photo. Please help me how to solve this python error and converting GEDI h5 to GeoJSON.
Attachments
3 python AttributeError.png
3 python AttributeError.png (112.22 KiB) Not viewed yet
2 --dir.png
2 --dir.png (52.03 KiB) Not viewed yet
1 --dir.png
1 --dir.png (22.46 KiB) Not viewed yet
Last edited by milansamanta on Sat Nov 25, 2023 6:47 am America/New_York, edited 1 time in total.

sbailey_kerber
Posts: 1
Joined: Wed Nov 29, 2023 12:27 am America/New_York
Answers: 0

Re: Converting GEDI h5 to GeoJSON

by sbailey_kerber » Wed Nov 29, 2023 12:53 am America/New_York

Hello,
I was getting the same error today and it looks like Line 177 (gediDF = gediDF.append(geoDF)) using the code append() has been depreciated and removed. This post explained it nicely: https://stackoverflow.com/questions/75956209/error-dataframe-object-has-no-attribute-append

Simply replacing the gediDF = gediDF.append(geoDF) with gediDF = gediDF.concat(geoDF) didn't work though and I still got this error:
File "C:\Users\sbailey-kerber\AppData\Local\ESRI\conda\envs\gedi\GEDI_Subsetter.py", line 177, in <module>
gediDF = gediDF.concat(geoDF)
File "C:\Users\sbailey-kerber\AppData\Local\ESRI\conda\envs\gedi\lib\site-packages\pandas\core\generic.py", line 6204, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'concat'

Does any one know a better code for line 177 gediDF = gediDF.append(geoDF) ?
In the meantime I'm going to try loading a previous version of pandas which will hopefully run the original append

LP DAAC - afriesz
Subject Matter Expert
Subject Matter Expert
Posts: 59
Joined: Tue Nov 12, 2019 4:02 pm America/New_York
Answers: 2
Been thanked: 2 times

Re: Converting GEDI h5 to GeoJSON

by LP DAAC - afriesz » Thu Nov 30, 2023 3:41 pm America/New_York

@sbailey_kerber ,

Thanks for reporting this. I just pushed out a fix to our GitHub repository https://github.com/nasa/GEDI-Data-Resources/blob/main/python/scripts/GEDI_Subsetter/GEDI_Subsetter.py. I've update the README.md as well, specifically the environment setup instructions https://github.com/nasa/GEDI-Data-Resources/blob/main/python/scripts/GEDI_Subsetter/README.md. Let us know if you continue to have problems.

Post Reply