Page 1 of 1

PIC netCDF files - scale and offset values for pic variable

Posted: Thu May 18, 2023 6:16 am America/New_York
by glastirmep
I have extracted daily PIC data for a couple of areas in the South Pacific through the Ocean Colour website. According to the variable attributes, the PIC scale_factor is 2.0e-6f and the add_offset 0.065f (I am uploading a screenshot from Panoply that shows the variable attributes). I have extracted the PIC values at specific locations using netCDF4 python library and they seem to have the scale_factor already applied (because of the magnitude of the values I am getting) but only the scale_factor, no the add_offset, even though I have not scripted that.
I have a couple of questions related to this:
1- Why am I extracting PIC values of the same order of magnitude as the scale_factor even though I have not apply this in my script? My understanding is that python netCDF4 library extracts the values as they are stored in the file without applying scale and offset unless scripted.
2 - From the variable attributes on the downloaded netCDF files, it seems that the add_offset value is several order of magnitude larger than the scale_factor and so, if applied, all PIC values are going to be equal to the add_offset value.
I would be very grateful if someone can help me understand these two issues I have described. I am happy to provide further information if my description is not clear.
Thank you very much.

Re: PIC netCDF files - scale and offset values for pic variable

Posted: Thu May 18, 2023 8:28 am America/New_York
by OB.DAAC - SeanBailey
The data are stored as signed short integers, so, if you are seeing floating point values, the function you used to read the PIC product applied the scale and offset values.

The scale_factor and add_offset values are applied to the data using this simple equation:

geophysical_value = scale_factor * scaled_integer_value + add_offset

The valid_min/valid_max attributes are the same data type as the data set (i.e. signed short integer). If we use the above equation, we can convert these attributes to their corresponding geophysical value:

valid_min = 2.0e-6 * -32500 + 0.065 ...which equals zero. If the add offset was not applied, the minimum value would be -0.065, which is not physically possible :D

BTW, the valid_max is 2.0e-6 * 30000 + 0.065, or 0.125.

*Most* of the PIC values are much lower than the valid_max, which was set to be sufficiently large to capture the range of data, but small enough to provide sufficient precision with scaling to an integer.

Hope this helps your understanding,

Re: PIC netCDF files - scale and offset values for pic variable

Posted: Thu May 25, 2023 12:02 pm America/New_York
by glastirmep
Yes, it does help. Thanks a lot for taking the time to reply to my post.