Page 1 of 1
Chlorophyll_a data retrieval
Posted: Fri Jun 04, 2021 1:37 pm America/New_York
by leon_c
Hello!
I have a question regarding downloading the data. Bear with me, I'm pretty new to this interface.
So, using the Level-3 Browser, I select the desired sensor, product, dates and extract the location.
However, if I download Binned file, once I open the .nc file in Python using xarray, I get a completely empty xarray.Dataset. If, on the other hand, I download Mapped file and repeat the proces in Python, I get chlor_a array of the desired shape (lat * lon), but all the data is nan (float 32)... I expected to have a float number for every pair of (lat,lon). Is this even how it's supposed to work?
Any tips and help would be much obliged! :)
Re: Chlorophyll_a data retrieval
Posted: Sun Jun 06, 2021 3:01 pm America/New_York
by leon_c
Figured it out. nan files was data over land, or missing data. Everything works fine. :)
Re: Chlorophyll_a data retrieval
Posted: Mon Jun 07, 2021 10:13 am America/New_York
by OB.DAAC - SeanBailey
Glad you figured it out
Just so you are aware, the bin files are not in a raster format. They can be read with Python, but a little bit of extra work is necessary to get to a recognizable image. There is an old, but still mostly relevant document on the binning process and format:
https://oceancolor.gsfc.nasa.gov/resources/docs/technical/seawifs_reports/prelaunch/vol32_abs/
A more recent dissertation on the subject is here:
https://oceancolor.gsfc.nasa.gov/resources/docs/format/l3bins/
Regards,
Sean
Re: Chlorophyll_a data retrieval
Posted: Mon Jun 07, 2021 11:23 am America/New_York
by leon_c
Thanks for your reply! I will look into it.
But the thing that worries me is, when the bin file is opened in Python using xarray, I get the result posted in the images. It's almost as if the file is "empty", but it's not.
Re: Chlorophyll_a data retrieval
Posted: Tue Jun 08, 2021 9:06 am America/New_York
by OB.DAAC - SeanBailey
Well, xarray is great for multi-dimensional array data, not so keen on the single dimensional compound data structure of the bin files. It can be read with Python and the netCDF4 module, but xarray probably can't interpret it out of the box. Here's what's contained in a typical bin file:
Code: Select all
group: level-3_binned_data {
types:
compound binListType {
uint bin_num ;
short nobs ;
short nscenes ;
float weights ;
float time_rec ;
}; // binListType
compound binDataType {
float sum ;
float sum_squared ;
}; // binDataType
compound binIndexType {
uint start_num ;
uint begin ;
uint extent ;
uint max ;
}; // binIndexType
dimensions:
binListDim = UNLIMITED ; // (318168 currently)
binDataDim = UNLIMITED ; // (318168 currently)
binIndexDim = UNLIMITED ; // (8640 currently)
variables:
binListType BinList(binListDim) ;
binDataType chlor_a(binDataDim) ;
binIndexType BinIndex(binIndexDim) ;
} // group level-3_binned_data
Some math is involved to convert the sum and the sum of squares into a mean value. More is required to identify where on the planet a bin is located given the bin index...
Sean
Re: Chlorophyll_a data retrieval
Posted: Mon Oct 30, 2023 3:25 pm America/New_York
by dominichr1
Did anyone find a way to read these binned files using xarray?
ds = xr.open_dataset(files[2], group="level-3_binned_data")
Re: Chlorophyll_a data retrieval
Posted: Thu Nov 02, 2023 3:43 pm America/New_York
by gnwiii
As this thread shows, writing software to work with binned data is a nontrivial project. You will need SeaDAS to validate any new implementation, so most people just use tools available in SeaDAS.