Crop and subset

cortivo · by **cortivo** » Thu Feb 16, 2017 4:18 pm America/New_York

I'm working with L3m (SMI) 8D SST Aqua and I'm cropping (subsetting) some specific regions through seadas (command line gpt ...). In the new netcdf file generated by seadas appears some things "crazy" ...:eek::eek::eek::confused:

When I read the global data from netcdf files using Matlab and perform the scaling factor I can obtain the same values of SST shown by seadas.

The new files (those with only subregions that were cropped by seadas and saved as netcdf4 format) I'm also reading them with Matlab and I note that the values of the SST are integer (ok) and negative(why?). Beyond that, when I apply the scaling and intercept factor in the SST (negative) values of the subregions read by Matlab I do not get the same values of the seadas.

What seadas do when it crop and save the subregions in a new file?

In attached figure is it possible to see the data saved in the netcdf files and read by matlab. One of them shows the sst values and the other one shows the sst_qual for the same pixels.

In figures I used this file: A20023212002328.L3m_8D_SST_sst_4km.nc and to do the crop I used N=-20; S=-21; W=-41; E=-40

gnwiii · by **gnwiii** » Fri Feb 17, 2017 8:04 am America/New_York

Matlab's current NetCDF reader should use the scale parameters (and missing value codes) from NetCDF-CF files without the need to apply scaling after the data have been loaded in Matlab. Older versions of Matlab did not handle some NetCDF-CF files ("unsupported data type") in which case the hdf5 reader can be used. In that case you should obtain the scaling parameters (and missing value codes) from the variable attributes because different software versions may use different storage formats and scaling.

cortivo · by **cortivo** » Fri Feb 17, 2017 11:33 am America/New_York

Ok.
But when I do the same thing as described before (with sst data), however with CHL data, I get from Matlab the same values of seadas.... I do not perform the same test with other data as FLH, KD490 and so on...

by **OB.DAAC - SeanBailey** » Fri Feb 17, 2017 12:03 pm America/New_York

The chlorophyll product is stored as a floating point value in the L3 SMI files, the sst product is stored as a scaled integer.
If Matlab is not applying the scale and offset to the scaled integers (although, George is correct in saying it should), you
will have to do so yourself to get the geophysical value from the scaled integer.

I followed the same steps you took and the resulting extracted file looks like this:

$ncdump A20023212002328.L3m_8D_SST_sst_4km
netcdf A20023212002328.L3m_8D_SST_sst_4km {
dimensions:
lat = 24 ;
lon = 24 ;
variables:
byte qual_sst(lat, lon) ;
qual_sst:coordinates = "lat lon" ;
qual_sst:flag_meanings = "Best Good Questionable Bad" ;
qual_sst:flag_masks = 0b, 1b, 2b, 3b ;
short sst(lat, lon) ;
sst:long_name = "Sea Surface Temperature" ;
sst:units = "degree_C" ;
sst:_Unsigned = "true" ;
sst:scale_factor = 0.000717184972018003 ;
sst:add_offset = -2. ;
sst:_FillValue = -1s ;
sst:coordinates = "lat lon" ;
float lat(lat) ;
lat:units = "degrees_north" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:valid_min = -20.97917f ;
lat:valid_max = -20.02083f ;
float lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
lon:valid_min = -40.97917f ;
lon:valid_max = -40.02084f ;

// global attributes:
:Conventions = "CF-1.4" ;
:TileSize = "24:24" ;
:start_date = "21-NOV-2002 22:17:34.742764" ;
:stop_date = "21-NOV-2002 23:20:17.061356" ;
:title = "A20023212002328.L3m_8D_SST_sst_4km.nc" ;
}

SeaDAS reads this file in and properly displays the values, so there is nothing amiss with the extraction.

Sean

cortivo · by **cortivo** » Fri Feb 17, 2017 12:57 pm America/New_York

If you read the data in netcdf files with ncdump you get

ncdump -v sst cropped_file.nc

Note that the sst data are all negative values

netcdf teste2 {
dimensions:
lat = 24 ;
lon = 24 ;
variables:
byte qual_sst(lat, lon) ;
qual_sst:coordinates = "lat lon" ;
qual_sst:flag_meanings = "Best Good Questionable Bad" ;
qual_sst:flag_masks = 0b, 1b, 2b, 3b ;
short sst(lat, lon) ;
sst:long_name = "Sea Surface Temperature" ;
sst:units = "degree_C" ;
sst:_Unsigned = "true" ;
sst:scale_factor = 0.000717184972018003 ;
sst:add_offset = -2. ;
sst:_FillValue = -1s ;
sst:coordinates = "lat lon" ;
float lat(lat) ;
lat:units = "degrees_north" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:valid_min = -20.97917f ;
lat:valid_max = -20.02083f ;
float lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
lon:valid_min = -40.97917f ;
lon:valid_max = -40.02084f ;

// global attributes:
:Conventions = "CF-1.4" ;
:TileSize = "24:24" ;
:start_date = "21-NOV-2002 22:17:34.742764" ;
:stop_date = "21-NOV-2002 23:20:17.061356" ;
:title = "A20023212002328.L3m_8D_SST_sst_4km.nc" ;
data:

sst =
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30396,
-29932, -29751,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30514,
-30183, -29545,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30793, -30366,
-29981, -29193,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -31103, -30797,
-30361, -29768, -29181,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -32305, -31110,
-30509, -29998, -29409,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -31932, -31932,
-31693, -30946, -30129, -29867,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -31486, -31678, _,
-29789, -29593, -29593,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30937, -32275, -32095,
-31575, -29849, -29456, -28923,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30897, -31999, -32046,
-31329, -30842, -29694, -29604,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -31375, -31464, -32269,
-32269, -31014, -31047, -30583, -29873,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -31227, -31280, -32300,
-30713, -30625, -30454, -29918, -29918,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -31294, -31123, -31637,
-30748, -30804, -30510, -29830, -29462,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30867, -30568, -30999,
-30495, -30466, -30164, -29638, -29298,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, -30568, -30581, -30770,
-30444, -30444, -30459, -30284, -29749, -29294,
_, _, _, _, _, _, _, _, _, _, _, _, _, _, -30005, -30456, -30629, -30118,
-30164, -30056, -29806, -29442, -29149, -29149,
_, _, _, _, _, _, _, _, _, _, _, _, _, -30723, -30544, -30231, -29360,
-29674, -30093, -29965, -30148, -29517, -29332, -28790,
_, _, _, _, _, _, _, _, _, _, _, -29485, -30255, -30326, -29751, -29751,
-29786, -29845, -29855, -29853, -29898, -29363, -29133, -28694,
_, _, _, _, _, _, _, _, _, _, _, -29805, -29936, -29778, -29437, -29469,
-29676, -29771, -29552, -29552, -29628, -29619, -29118, -28822,
_, _, _, _, _, _, _, _, _, _, -29555, -29482, -29459, -29387, -29682,
-29285, -29309, -29630, -29586, -29530, -29846, -29613, -29613, -29250,
_, _, _, _, _, _, _, _, _, -30311, -30000, -30000, -29735, -29739, -29759,
-29386, -29413, -29701, -29300, -29512, -29747, -29618, -29319, -28534,
_, _, _, _, _, _, _, -28577, -29130, -29416, -29631, -29531, -29681,
-29700, -29548, -29548, -29530, -29589, -29173, -29268, -29304, -29429,
-29229, -28409,
_, _, _, _, _, _, -28827, -29006, -29460, -29600, -29599, -29904, -29783,
-29478, -29403, -29309, -29511, -29494, -29434, -29434, -29491, -29533,
-29320, -28729,
_, _, _, _, -28990, -28989, -28890, -28890, -29139, -29387, -29616, -29959,
-29645, -29548, -29555, -29786, -29670, -29326, -29169, -29509, -29267,
-29029, -29029, -28213,
_, _, _, _, -28350, -29223, -29148, -29328, -29640, -29716, -29911, -29911,
-29864, -29575, -29471, -29523, -29573, -29450, -29229, -29386, -28894,
-28236, -27859, -27620 ;
}

My question is why the values are changed to negative values....

Fabio.

by **OB.DAAC - SeanBailey** » Fri Feb 17, 2017 1:21 pm America/New_York

Because they are scaled integers.

sst = scale_factor * scaled integer + add_offset

gnwiii · by **gnwiii** » Fri Feb 17, 2017 2:08 pm America/New_York

The negative values you are seeing from ncdump occur because the "unsigned" attribute is not be respected. The NetCDF header says

short sst(lat, lon) ;
    sst:long_name = "Sea Surface Temperature" ;
    sst:units = "degree_C" ;
    sst:_Unsigned = "true" ;
    sst:scale_factor = 0.000717184972018003 ;
    sst:add_offset = -2. ;
    sst:_FillValue = -1s ;
    sst:coordinates = "lat lon" ;

Note that you have unsigned short integer data. From [url=http://www.unidata.ucar.edu/software/netcdf/docs/data_type.html]NetCDF External Data Types/url], unsigned short support was added with NetCDF4, so older software probable treats the values as signed shorts. IDL/GDL ints are what NetCDF calls "short", so:

GDL> i=UINT(-30396)    
print, i, i*0.000717184972018003 - 2
   35140      23.2019