HICO level 1b data - true_color SDS

[amsayer]

I've successfully got some HICO level 1b data from your server. As a sanity check, I am trying to reproduce your true-colour quicklook image for the scene I have got (H2011140064744.L1B_ISS).

I noticed an SDS images/true_color, which looked like the place to start. I read this in successfully as a 3x512x2000 byte array. However when I image the array, something looks off: it looks like each wavelength slice [0-2,*,*] contains a mini-image repeated 9 times (in a 3x3 pattern). Each wavelength is different, i.e. [0,*,*] differs from [1,*,*], but they're all repeated images. So clearly I am doing something incorrectly... I was hoping you could point me towards a user guide for these files (I did not spot one) or be able to point out where I am going on? I'm working in IDL.

Note I have already managed to read in the latitude/longitude and solar/sensor geometries and they look as expected, it's just the true_color SDS I am having difficulties with so far. (I did not look at the raw radiances yet--first things first.) Any guidance would be appreciated!

Thanks,

Andy

[amsayer]

As an update, I'm able to read the radiances ok and get a sensible true colour image from them - it's just the existing true_color SDS which is confounding me.

[OB WebDev - norman]

Hi Andy,

The "/images/true_color" data set is stored
band sequentially.  The nine panels you see
would result if you tried to view the data via
a format that expected the red, green, and blue
values for each pixel to be adjacent to each other.

I am unfamiliar with IDL, but I do often work with
the PPM image format which does have red,
green, and blue values adjacent.  The following
h5dump and perl snippets would represent the
true_color image data correctly.

h5dump -o H2011140064744.L1B_ISS.true_color.dat \
            -d /images/true_color
            -b NATIVE  H2011140064744.L1B_ISS

perl -e 'open F,"H2011140064744.L1B_ISS.true_color.dat";' \
     -e 'read F,$r,1024000;' \
     -e 'read F,$g,1024000;' \
     -e 'read F,$b,1024000;' \
     -e 'close F;' \
     -e 'print "P6\n512 2000\n255\n";' \
     -e 'for($i=0;$i<1024000;$i++){' \
     -e '  print substr($r,$i,1),substr($g,$i,1),substr($b,$i,1);' \
     -e '}' \
> H2011140064744.L1B_ISS.true_color.ppm

cjpeg H2011140064744.L1B_ISS.true_color.ppm \
> H2011140064744.L1B_ISS.true_color.jpg

I attach the resulting image (converted to JPEG) below.

I hope this helps.

Norman

[OB WebDev - norman]

P.S.  Note that our browse images are not reproductions
       of the /images/true_color data set.

[amsayer]

Hi Norman, thanks for the reply. I will see if I can find some transform which allows things to come out correctly. But if you do an h5dump of the file it does suggest it is supposed to be a 3 by x by y array:

   GROUP "images" {
      DATASET "true_color" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SIMPLE { ( 2000, 512, 3 ) / ( 2000, 512, 3 ) }
         ATTRIBUTE "CLASS" {
            DATATYPE  H5T_STRING {
                  STRSIZE 6;
                  STRPAD H5T_STR_NULLTERM;
                  CSET H5T_CSET_ASCII;
                  CTYPE H5T_C_S1;
               }
            DATASPACE  SCALAR
         }

So that would be why I got confused!

[OB WebDev - norman]

Yes, those metadata are misleading since they
give the wrong order of the dimensions.  Given
the way the data are organized, the dataspace
dimensions should read ( 3, 2000, 512 ).  I will
pass this along to the L1B-file creators in hopes
of a fix.

Regards,
Norman

[OB WebDev - norman]

Andy,

It turns out that our group runs the code
that makes these files, but we are not
the authors of the code and do not have
access to the sources.  We have requested
the source code, but the HICO team is
understandably more concerned with getting
the sensor turned back on at the moment.

Norman

[amsayer]

Thanks, Norman. And yes, I hope that they are able to get it working again!

[amsayer]

Just wanted to post that this is some code to reorder the array and produce an equivalent to Norman's, in IDL:

d = h5d_open(fid,'images/true_color')
tc = H5D_READ(d)

npts=512l*2000l ; Size of the granule

tc2=tc
tc2[0,*,*]=tc[0:npts-1]
tc2[1,*,*]=tc[npts:2*npts-1]
tc2[2,*,*]=tc[2*npts:3*npts-1]

tc=tc2
tc2=0