HICO level 1b data - true_color SDS

Please enter here to ask a question about any NASA Science related topics!
Post Reply
amsayer
Posts: 22
Joined: Tue Jan 25, 2011 3:03 pm America/New_York

HICO level 1b data - true_color SDS

by amsayer » Thu Jan 16, 2014 11:52 am America/New_York

I've successfully got some HICO level 1b data from your server. As a sanity check, I am trying to reproduce your true-colour quicklook image for the scene I have got (H2011140064744.L1B_ISS).

I noticed an SDS images/true_color, which looked like the place to start. I read this in successfully as a 3x512x2000 byte array. However when I image the array, something looks off: it looks like each wavelength slice [0-2,*,*] contains a mini-image repeated 9 times (in a 3x3 pattern). Each wavelength is different, i.e. [0,*,*] differs from [1,*,*], but they're all repeated images. So clearly I am doing something incorrectly... I was hoping you could point me towards a user guide for these files (I did not spot one) or be able to point out where I am going on? I'm working in IDL.

Note I have already managed to read in the latitude/longitude and solar/sensor geometries and they look as expected, it's just the true_color SDS I am having difficulties with so far. (I did not look at the raw radiances yet--first things first.) Any guidance would be appreciated!

Thanks,

Andy

Tags:

amsayer
Posts: 22
Joined: Tue Jan 25, 2011 3:03 pm America/New_York

HICO level 1b data - true_color SDS

by amsayer » Thu Jan 16, 2014 1:09 pm America/New_York

As an update, I'm able to read the radiances ok and get a sensible true colour image from them - it's just the existing true_color SDS which is confounding me.

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 136
Joined: Tue Feb 09, 2021 8:19 am America/New_York

HICO level 1b data - true_color SDS

by OB WebDev - norman » Thu Jan 16, 2014 1:48 pm America/New_York

Hi Andy,

The "/images/true_color" data set is stored
band sequentially.  The nine panels you see
would result if you tried to view the data via
a format that expected the red, green, and blue
values for each pixel to be adjacent to each other.

I am unfamiliar with IDL, but I do often work with
the PPM image format which does have red,
green, and blue values adjacent.  The following
h5dump and perl snippets would represent the
true_color image data correctly.

h5dump -o H2011140064744.L1B_ISS.true_color.dat \
            -d /images/true_color
            -b NATIVE  H2011140064744.L1B_ISS

perl -e 'open F,"H2011140064744.L1B_ISS.true_color.dat";' \
     -e 'read F,$r,1024000;' \
     -e 'read F,$g,1024000;' \
     -e 'read F,$b,1024000;' \
     -e 'close F;' \
     -e 'print "P6\n512 2000\n255\n";' \
     -e 'for($i=0;$i<1024000;$i++){' \
     -e '  print substr($r,$i,1),substr($g,$i,1),substr($b,$i,1);' \
     -e '}' \
> H2011140064744.L1B_ISS.true_color.ppm

cjpeg H2011140064744.L1B_ISS.true_color.ppm \
> H2011140064744.L1B_ISS.true_color.jpg

I attach the resulting image (converted to JPEG) below.

I hope this helps.

Norman

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 136
Joined: Tue Feb 09, 2021 8:19 am America/New_York

HICO level 1b data - true_color SDS

by OB WebDev - norman » Thu Jan 16, 2014 1:54 pm America/New_York

P.S.  Note that our browse images are not reproductions
       of the /images/true_color data set.

amsayer
Posts: 22
Joined: Tue Jan 25, 2011 3:03 pm America/New_York

HICO level 1b data - true_color SDS

by amsayer » Thu Jan 16, 2014 3:46 pm America/New_York

Hi Norman, thanks for the reply. I will see if I can find some transform which allows things to come out correctly. But if you do an h5dump of the file it does suggest it is supposed to be a 3 by x by y array:

   GROUP "images" {
      DATASET "true_color" {
         DATATYPE  H5T_STD_U8LE
         DATASPACE  SIMPLE { ( 2000, 512, 3 ) / ( 2000, 512, 3 ) }
         ATTRIBUTE "CLASS" {
            DATATYPE  H5T_STRING {
                  STRSIZE 6;
                  STRPAD H5T_STR_NULLTERM;
                  CSET H5T_CSET_ASCII;
                  CTYPE H5T_C_S1;
               }
            DATASPACE  SCALAR
         }

So that would be why I got confused!

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 136
Joined: Tue Feb 09, 2021 8:19 am America/New_York

HICO level 1b data - true_color SDS

by OB WebDev - norman » Thu Jan 16, 2014 4:11 pm America/New_York

Yes, those metadata are misleading since they
give the wrong order of the dimensions.  Given
the way the data are organized, the dataspace
dimensions should read ( 3, 2000, 512 ).  I will
pass this along to the L1B-file creators in hopes
of a fix.

Regards,
Norman

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 136
Joined: Tue Feb 09, 2021 8:19 am America/New_York

HICO level 1b data - true_color SDS

by OB WebDev - norman » Thu Jan 16, 2014 4:40 pm America/New_York

Andy,

It turns out that our group runs the code
that makes these files, but we are not
the authors of the code and do not have
access to the sources.  We have requested
the source code, but the HICO team is
understandably more concerned with getting
the sensor turned back on at the moment.

Norman

amsayer
Posts: 22
Joined: Tue Jan 25, 2011 3:03 pm America/New_York

HICO level 1b data - true_color SDS

by amsayer » Fri Jan 17, 2014 8:36 am America/New_York

Thanks, Norman. And yes, I hope that they are able to get it working again!

amsayer
Posts: 22
Joined: Tue Jan 25, 2011 3:03 pm America/New_York

HICO level 1b data - true_color SDS

by amsayer » Fri Jan 17, 2014 10:10 am America/New_York

Just wanted to post that this is some code to reorder the array and produce an equivalent to Norman's, in IDL:

d = h5d_open(fid,'images/true_color')
tc = H5D_READ(d)

npts=512l*2000l ; Size of the granule

tc2=tc
tc2[0,*,*]=tc[0:npts-1]
tc2[1,*,*]=tc[npts:2*npts-1]
tc2[2,*,*]=tc[2*npts:3*npts-1]

tc=tc2
tc2=0

Post Reply