HICO level 1b data - true_color SDS
HICO level 1b data - true_color SDS
I've successfully got some HICO level 1b data from your server. As a sanity check, I am trying to reproduce your true-colour quicklook image for the scene I have got (H2011140064744.L1B_ISS).
I noticed an SDS images/true_color, which looked like the place to start. I read this in successfully as a 3x512x2000 byte array. However when I image the array, something looks off: it looks like each wavelength slice [0-2,*,*] contains a mini-image repeated 9 times (in a 3x3 pattern). Each wavelength is different, i.e. [0,*,*] differs from [1,*,*], but they're all repeated images. So clearly I am doing something incorrectly... I was hoping you could point me towards a user guide for these files (I did not spot one) or be able to point out where I am going on? I'm working in IDL.
Note I have already managed to read in the latitude/longitude and solar/sensor geometries and they look as expected, it's just the true_color SDS I am having difficulties with so far. (I did not look at the raw radiances yet--first things first.) Any guidance would be appreciated!
Thanks,
Andy
I noticed an SDS images/true_color, which looked like the place to start. I read this in successfully as a 3x512x2000 byte array. However when I image the array, something looks off: it looks like each wavelength slice [0-2,*,*] contains a mini-image repeated 9 times (in a 3x3 pattern). Each wavelength is different, i.e. [0,*,*] differs from [1,*,*], but they're all repeated images. So clearly I am doing something incorrectly... I was hoping you could point me towards a user guide for these files (I did not spot one) or be able to point out where I am going on? I'm working in IDL.
Note I have already managed to read in the latitude/longitude and solar/sensor geometries and they look as expected, it's just the true_color SDS I am having difficulties with so far. (I did not look at the raw radiances yet--first things first.) Any guidance would be appreciated!
Thanks,
Andy
Tags:
HICO level 1b data - true_color SDS
As an update, I'm able to read the radiances ok and get a sensible true colour image from them - it's just the existing true_color SDS which is confounding me.
-
- Subject Matter Expert
- Posts: 142
- Joined: Tue Feb 09, 2021 8:19 am America/New_York
HICO level 1b data - true_color SDS
Hi Andy,
The "/images/true_color" data set is stored
band sequentially. The nine panels you see
would result if you tried to view the data via
a format that expected the red, green, and blue
values for each pixel to be adjacent to each other.
I am unfamiliar with IDL, but I do often work with
the PPM image format which does have red,
green, and blue values adjacent. The following
h5dump and perl snippets would represent the
true_color image data correctly.
I attach the resulting image (converted to JPEG) below.
I hope this helps.
Norman
The "/images/true_color" data set is stored
band sequentially. The nine panels you see
would result if you tried to view the data via
a format that expected the red, green, and blue
values for each pixel to be adjacent to each other.
I am unfamiliar with IDL, but I do often work with
the PPM image format which does have red,
green, and blue values adjacent. The following
h5dump and perl snippets would represent the
true_color image data correctly.
h5dump -o H2011140064744.L1B_ISS.true_color.dat \
-d /images/true_color
-b NATIVE H2011140064744.L1B_ISS
perl -e 'open F,"H2011140064744.L1B_ISS.true_color.dat";' \
-e 'read F,$r,1024000;' \
-e 'read F,$g,1024000;' \
-e 'read F,$b,1024000;' \
-e 'close F;' \
-e 'print "P6\n512 2000\n255\n";' \
-e 'for($i=0;$i<1024000;$i++){' \
-e ' print substr($r,$i,1),substr($g,$i,1),substr($b,$i,1);' \
-e '}' \
> H2011140064744.L1B_ISS.true_color.ppm
cjpeg H2011140064744.L1B_ISS.true_color.ppm \
> H2011140064744.L1B_ISS.true_color.jpg
I attach the resulting image (converted to JPEG) below.
I hope this helps.
Norman

-
- Subject Matter Expert
- Posts: 142
- Joined: Tue Feb 09, 2021 8:19 am America/New_York
HICO level 1b data - true_color SDS
P.S. Note that our browse images are not reproductions
of the /images/true_color data set.
of the /images/true_color data set.
HICO level 1b data - true_color SDS
Hi Norman, thanks for the reply. I will see if I can find some transform which allows things to come out correctly. But if you do an h5dump of the file it does suggest it is supposed to be a 3 by x by y array:
GROUP "images" {
DATASET "true_color" {
DATATYPE H5T_STD_U8LE
DATASPACE SIMPLE { ( 2000, 512, 3 ) / ( 2000, 512, 3 ) }
ATTRIBUTE "CLASS" {
DATATYPE H5T_STRING {
STRSIZE 6;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
So that would be why I got confused!
GROUP "images" {
DATASET "true_color" {
DATATYPE H5T_STD_U8LE
DATASPACE SIMPLE { ( 2000, 512, 3 ) / ( 2000, 512, 3 ) }
ATTRIBUTE "CLASS" {
DATATYPE H5T_STRING {
STRSIZE 6;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
So that would be why I got confused!
-
- Subject Matter Expert
- Posts: 142
- Joined: Tue Feb 09, 2021 8:19 am America/New_York
HICO level 1b data - true_color SDS
Yes, those metadata are misleading since they
give the wrong order of the dimensions. Given
the way the data are organized, the dataspace
dimensions should read ( 3, 2000, 512 ). I will
pass this along to the L1B-file creators in hopes
of a fix.
Regards,
Norman
give the wrong order of the dimensions. Given
the way the data are organized, the dataspace
dimensions should read ( 3, 2000, 512 ). I will
pass this along to the L1B-file creators in hopes
of a fix.
Regards,
Norman
-
- Subject Matter Expert
- Posts: 142
- Joined: Tue Feb 09, 2021 8:19 am America/New_York
HICO level 1b data - true_color SDS
Andy,
It turns out that our group runs the code
that makes these files, but we are not
the authors of the code and do not have
access to the sources. We have requested
the source code, but the HICO team is
understandably more concerned with getting
the sensor turned back on at the moment.
Norman
It turns out that our group runs the code
that makes these files, but we are not
the authors of the code and do not have
access to the sources. We have requested
the source code, but the HICO team is
understandably more concerned with getting
the sensor turned back on at the moment.
Norman
HICO level 1b data - true_color SDS
Thanks, Norman. And yes, I hope that they are able to get it working again!
HICO level 1b data - true_color SDS
Just wanted to post that this is some code to reorder the array and produce an equivalent to Norman's, in IDL:
d = h5d_open(fid,'images/true_color')
tc = H5D_READ(d)
npts=512l*2000l ; Size of the granule
tc2=tc
tc2[0,*,*]=tc[0:npts-1]
tc2[1,*,*]=tc[npts:2*npts-1]
tc2[2,*,*]=tc[2*npts:3*npts-1]
tc=tc2
tc2=0
d = h5d_open(fid,'images/true_color')
tc = H5D_READ(d)
npts=512l*2000l ; Size of the granule
tc2=tc
tc2[0,*,*]=tc[0:npts-1]
tc2[1,*,*]=tc[npts:2*npts-1]
tc2[2,*,*]=tc[2*npts:3*npts-1]
tc=tc2
tc2=0