Page 2 of 3
Posted: Thu Feb 11, 2021 2:46 pm America/New_York
I am doing mosaic of a few mages and want to set certain value to NaN
I followed this statement in .par file:
band1Expression=if (Kd_490 != 0) then 1.3/Kd_490 else NaN
and have a statement that looks like:
band1Expression=if (rhos_443 != 0) then rhos_443 else NaN
But I still get 0s and not NaN.
Not sure what I am doing wrong.
Posted: Thu Feb 11, 2021 6:55 pm America/New_York
You need to provide enough information so we can easily reproduce the problem. It would be useful to know the file format you are using and whether NaN's are in locations that were not valid data in the input file (a report in the SNAP forum
of scattered NaN's using gpt graph processing and writing to NetCDF4-BEAM that suggests "valid" pixels are being converted to NaN's).
In the standard SNPP_VIIRS level-2 OC NetCDF files, Kd_490 not-valid values display as NaN's in the SeaDAS GUI using the "Pixel Info" panel, but the band Properties show:
Valid-Pixel Expression Kd_490 >= 0.01000 && Kd_490 <= 6.00000
No-Data Value Used true
No-Data Value -32767.0
In the NetCDF metadate these translate to:
short Kd_490(number_of_lines, pixels_per_line) ;
Kd_490:long_name = "Diffuse attenuation coefficient at 490 nm, KD2 algorithm" ;
Kd_490:scale_factor = 0.0002f ;
Kd_490:add_offset = 0.f ;
Kd_490:units = "m^-1" ;
Kd_490:_FillValue = -32767s ;
Kd_490:valid_min = 50s ;
Kd_490:valid_max = 30000s ;
The valid pixel expression for Kd_490 is based on
, but the entry in the same file for
is missing the
values. NetCDF readers often translate the
to NaN when loading data and applying the offset and scale.
Posted: Fri Feb 12, 2021 9:44 am America/New_York
Thank you for the explanation. What I want to do is exclude land and cloud pixels from mosaic/composite and then set rhos to NaN or some missing value number if the value is 0. My par file looks like this:
band1Expression=if (rhos_412 != 0) then rhos_412 else NaN
qualityExpression=!l2_flags.LAND and !l2_flags.CLDICE
crs=GEOGCS["WGS84(DD)", DATUM["WGS84", SPHEROID["WGS84", 6378137.0, 298.257223563]], PRIMEM["Greenwich", 0.0], UNIT["degree", 0.017453292519943295], AXIS["Geodetic longitude", EAST], AXIS["Geodetic latitude", NORTH]]
And the xml file :
Posted: Fri Feb 12, 2021 10:43 am America/New_York
We still don't know what file formats you are using (metadata handling is not consistent across formats) and how the NaN values in the output images relate to missing or flagged pixels in the level-2 file. Can you select a small area in the output image with NaN and valid pixels? If so you can inspect the same region in the input level-2 files to get an understanding of what triggers the NaN values. It might be helpful to modify the
product.xml file to add the valid min and max for entries for rhos so these appear in the GUI properties window.
Posted: Fri Feb 12, 2021 12:30 pm America/New_York
In the GUI I can see NaN where there is no data (Rhos =0 there show up as NaN) it is the NetCDF file that they stay 0s and no appear as NaN. I tried a number (-9999) instead of NaN but that does not work either!
Thanks much for your help.
Posted: Fri Feb 12, 2021 5:18 pm America/New_York
Where are you seeing rhos=0? In the
calc_rhos.c source I see
l1rec->rhos[ipb] = BAD_FLT; unless
l1rec->Lt[ipb] <= 0., where
#define BAD_FLT -32767.0.
Posted: Sat Feb 13, 2021 1:23 pm America/New_York
I just look at ncdump of the L2 nc file.
Posted: Sat Feb 13, 2021 5:08 pm America/New_York
I used the SeaDAS benchmark files (A2006167181000.L1B and A2006167181000.GEO) to get rhos_443. ncdump -h shows:
float rhos_443(number_of_lines, pixels_per_line) ;
rhos_443:long_name = "Surface reflectance at 443 nm" ;
rhos_443:standard_name = "surface_albedo" ;
rhos_443:_FillValue = -32767.f ;
rhos_443:valid_min = -0.05f ;
rhos_443:valid_max = 1.f ;
so zero is considered "valid".
Posted: Mon Feb 15, 2021 10:04 am America/New_York
Thanks. Here is a portion of what I get after I use gpt to mosaic files:
metadata:Processing_Graph:node_0:parameters:orthorectify = "false" ;
metadata:Processing_Graph:node_0:parameters:combine = "OR" ;
metadata:Processing_Graph:node_0:parameters:variables:variable:name = "rhos_412" ;
metadata:Processing_Graph:node_0:parameters:variables:variable:expression = "if (rhos_412 != 0) then rhos_412 else -999" ;
metadata:Processing_Graph:node_0:parameters:conditions:condition:name = "condition_0" ;
metadata:Processing_Graph:node_0:parameters:conditions:condition:output = "false" ;
metadata:Processing_Graph:node_0:parameters:conditions:condition:expression = "!l2_flags.LAND and !l2_flags.CLDICE" ;
float rhos_412(lat, lon) ;
rhos_412:long_name = "if (rhos_412 != 0) then rhos_412 else -999" ;
rhos_412:coordinates = "lat lon" ;
rhos_412:valid_pixel_expression = "rhos_412_count > 0" ;
All I wan to do is set '0' values to some number or NaN.
Posted: Mon Feb 15, 2021 11:07 am America/New_York
Try looking at the histogram in the SeaDAS GUI. My test image has only 6 values below 0.015, and no zeros. In the GUI, the FillValue (-32767.0) is translated to NaN, but if you are using a different file format or reader the FillValue may be left as -32767.0.