Extract L1A MODIS SeaDAS 8.1.0 OCSSW

Please enter here to ask a question about any NASA Science related topics!
daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by daurin » Thu Nov 04, 2021 10:40 am America/New_York

I've been having trouble with the OCSSW L1A MODIS extractor. Using the GUI, entering the Lon/Lat for extraction does not propagate to Pixels as expected and the extractor fails with:

execution exception: java.io.IOException: l1aextract_modis failed with exit code 1.
Check log for more details.
This is version 0.76 of l1aextract_modis (compiled on Jun 18 2021 11:20:21)

Usage: l1aextract_modis infile spixl epixl sline eline outfile
where:
infile - input MODIS L1A datafile
spixl - start pixel number (1-based)
epixl - end pixel number (1-based)
sline - start line (1-based)
eline - end line (1-based)
outfile - output file name

Note: Enter line number NOT scan number!

When I try to get the pixline data to manually enter into the GUI, it fails:

daurin@GSSLA18091785:~/Projects_Supplemental/Commercial_Sat/Satellite_Imagery/MODIS$ lonlat2pixline T2021113153500.L1A_LAC T2021113153500.GEO -79.48 39.0 -75.9 39.6
-E- /Users/dshea/focs/oel_hdf4/filetype/filetype.c Line 692: Unrecognized HDF-EOS file T2021113153500.L1A_LAC
-E- lonlat2pixline: File type not recognized for T2021113153500.L1A_LAC.

And yet SeaDAS had no trouble reading the L1A_LAC or creating a GEO file for it. That log ends with:

Validation test passed for geolocation file /Users/daurin/Projects_Supplemental/Commercial_Sat/Satellite_Imagery/MODIS/T2021113153500.GEO
geogen_modis created /Users/daurin/Projects_Supplemental/Commercial_Sat/Satellite_Imagery/MODIS/T2021113153500.GEO successfully!
MODIS geolocation processing complete.

No log was created by extractl1A_modis that I can see.

Dirk

Tags:

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by daurin » Thu Nov 04, 2021 12:01 pm America/New_York

(I meant l1aextract_modis, of course. Also, this appears similar to an older problem: https://oceancolor.gsfc.nasa.gov/forum/ ... hl=extract)

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 200
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by OB SeaDAS - xuanyang02 » Thu Nov 04, 2021 2:45 pm America/New_York

Dirk,

It seems that lonlat2pixline has some issues with terra files. It works with aqua files:
$ lonlat2pixline A2010283180500.L1B_LAC A2010283180500.GEO -78.0 37.0 -75.0 39.0
# 701 985 1023 1289
sline=1023
eline=1289
spixl=701
epixl=985

I used 1071 1287 203 597 for spixl epixl sline eline on Extractor GUI, and l1aextract_modis seems to work and generated TERRA_MODIS.20210423T153500.sub. I downloaded the corresponding L2 files and estimated the spixl, epixl, sline, and eline.

Bing

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by daurin » Thu Nov 04, 2021 3:25 pm America/New_York

Thanks, Bing. Unfortunately, I am having the same problem with Aqua files: failure to autofill the Pixels fields in the extractor GUI, and failure to run lonlat2pixline.

lonlat2pixline A2021157174000.L1A_LAC A2021157174000.GEO -76.5 39.2 -76 39.6
-E- /Users/dshea/focs/oel_hdf4/filetype/filetype.c Line 692: Unrecognized HDF-EOS file A2021157174000.L1A_LAC
-E- lonlat2pixline: File type not recognized for A2021157174000.L1A_LAC.

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 200
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by OB SeaDAS - xuanyang02 » Thu Nov 04, 2021 3:31 pm America/New_York

Dirk,

I forgot to mention lonlat2pixline needs an L1B file for input. I was also able to do lonlat2pixline on the terra file:

$ lonlat2pixline T2021113153500.L1B_LAC T2021113153500.GEO -76.5 39.2 -76 39.6
# 245 275 1024 1072
sline=1024
eline=1072
spixl=245
epixl=275


Bing
Last edited by OB SeaDAS - xuanyang02 on Thu Nov 04, 2021 3:47 pm America/New_York, edited 1 time in total.

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by daurin » Thu Nov 04, 2021 3:47 pm America/New_York

I see. And yet, that raises another (possibly related?) issue. In all the OCSSW guis, the Load Parameters/Save Parameters and update luts buttons are greyed out. When I try to process that Aqua file to L1B, I get:

execution exception: java.io.IOException: modis_L1B failed with exit code 1.
Check log for more details.
MODIS Aqua L1B version 6.2.1_obpg, built Jun 18 2021 11:20:25

LUT directory: /Users/daurin/Applications/seadas-8.1.0/ocssw/var/modisa/cal/OPER
LUT version: 6.2.1.57_OC_v1.57
Reflective LUT: MYD02_Reflective_LUTs.V6.2.1.57_OC_v1.57.hdf
Emissive LUT: MYD02_Emissive_LUTs.V6.2.1.57_OC_v1.57.hdf
QA LUT: MYD02_QA_LUTs.V6.2.1.57_OC_v1.57.hdf

Processing MODIS L1A file to L1B...
/Users/daurin/Applications/seadas-8.1.0/ocssw/bin/l1bgen_modisa exit status: 1
ERROR: MODIS L1B processing failed.
Please examine the LogStatus and LogUser files for more information.

No new log file was produced for modis_L1B in this case.

I believe these are new problems since upgrading to Big Sur (11.6), unlike the extractor issue which predated the OS upgrade.

OB SeaDAS - xuanyang02
Subject Matter Expert
Subject Matter Expert
Posts: 200
Joined: Tue Feb 09, 2021 5:42 pm America/New_York
Answers: 1

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by OB SeaDAS - xuanyang02 » Thu Nov 04, 2021 6:03 pm America/New_York

Extractor actually works if you feed spixl, epixl, sline and eline to it. If you input lon/lat, you need to click "apply" button for pixel fields to be populated. Also, lonlat2pixline works if you feed L1B files to it.

My mac is Big Sur 11.6.

Regarding modis_L1B, are you trying to process A2010283180500.L1A_LAC to L1B?

I was able to take A2010283180500.L1A_LAC as input, and update luts button was not greyed out. I was also able to generate A2010283180500.L1B_LAC

The log file shows:
MODIS Aqua L1B version 6.2.1_obpg, built Jun 18 2021 11:20:25
...
LUT directory: /Users/bingyang/seadas-8.1.0/ocssw/var/modisa/cal/OPER
LUT version: 6.2.1.60_OC_v1.60
Reflective LUT: MYD02_Reflective_LUTs.V6.2.1.60_OC_v1.60.hdf
Emissive LUT: MYD02_Emissive_LUTs.V6.2.1.60_OC_v1.60.hdf
QA LUT: MYD02_QA_LUTs.V6.2.1.60_OC_v1.60.hdf

Processing MODIS L1A file to L1B...
/Users/bingyang/seadas-8.1.0/ocssw/bin/l1bgen_modisa exit status: 0
MODIS L1B processing complete.

Can you do SeaDAS/System Info and post the results here?

daurin
Posts: 27
Joined: Thu Apr 22, 2010 8:06 am America/New_York

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by daurin » Fri Nov 05, 2021 2:33 pm America/New_York

Hi Bing,

When extractor failed to extract A2021157174000.L1A_LAC, and lonlat2pixline could only read L1B, I tried converting it to L1B yesterday with the results above. Despite the error, it produced L1B_QKM and L1B_HKM files, but rather than a L1B_LAC, it created something I've never seen before, an L1B_OBC file (A2021157174000.L1B_OBC). I restarted and tried again fresh today and the L1A>L1B processor worked this time without error. Go figure?

Nevertheless, the extractor still does not propagate Lon/Lat values into the Pixels fields, regardless of hitting Apply. Load and Save Parameters are still grayed out (as is outfile, incidentally, but I think that might be normal?)

However, now that I have the L1B, I can get the Pixels info:
lonlat2pixline A2021157174000.L1B_LAC A2021157174000.GEO -76.5 39.2 -76.0 39.6
# 1163 1189 1297 1351
sline=1297
eline=1351
spixl=1163
epixl=1189

Using these in the extractor on the L1A file appeared to work, but when processed up to L2, it's the wrong region (and must be over all cloud). Even if it had worked that way, the approach sort of defeats my objective to reduce processing time and disk usage by subsetting prior to processing L1B/L2.

Sorry, I'm not sure what you mean by SeaDAS/System Info, but I attached the results of About (for some reason the text from that window can not be selected to copy/paste here). The system is macOS Big Sur 11.6.

I also attached the result showing the uncropped L1B zoomed over my AOI together with the extractor result processed to L2 as per above.

Dirk
Attachments
seadas8_a.png
seadas8_a.png (340.09 KiB) Not viewed yet
seadas8.png
seadas8.png (89.08 KiB) Not viewed yet
Last edited by daurin on Fri Nov 05, 2021 2:56 pm America/New_York, edited 1 time in total.

gnwiii
Posts: 654
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by gnwiii » Fri Nov 05, 2021 2:51 pm America/New_York

daurin wrote: Fri Nov 05, 2021 2:33 pm America/New_York Using these in the extractor on the L1A file appeared to work, but when processed up to L2, it's the wrong region (and must be over all cloud). Even if it had worked that way, the approach sort of defeats my objective to reduce processing time and disk usage by subsetting prior to processing L1B/L2.
l2gen (on a full granual) is the time-consuming step. SeaDAS itself has a low memory footprint by modern standards (the original design dates back to the 1990's when memory was a scarce resource). On a modern linux system you may want to create a ramdisk for temporary storage of intermediate files that are used just once.
daurin wrote: Fri Nov 05, 2021 2:33 pm America/New_York Sorry, I'm not sure what you mean by SeaDAS/System Info, but I attached the results of About (for some reason the text from that window can not be selected to copy/paste here.
"SeaDAS/System Info" means use the "SeaDAS-OCSSW" tab to run "System Info". It has much more detail and has a "copy to clipboard" button -- ideal for forum posts.

[Edit]When in doubt, use the search tool in the Help system.

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1233
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1

Re: Extract L1A MODIS SeaDAS 8.1.0 OCSSW

by OB.DAAC - SeanBailey » Mon Nov 08, 2021 9:05 am America/New_York

Dirk,

The OBC file is always created by the L1B code, but by default the wrapper script deletes it...unless you check the "keep-obc" box (or pass the script --keep-obc).
If you accidentally checked the obc box AND the del_okm box (which tells the script to delete the 1km output file - the L1B_LAC file), then you'd see the result you report.

I used the extract parameters you provided for the file you listed, and the extractor produced exactly what is was asked to do - the region extracted is correct.

Yes, this process will do what you seek (reducing processing time and disk storage).
Yes, you need to generate a full GEO and L1B file first, but those files are temporary (and it's a quick process). Once you've generated the extracted L1A, you no longer need the full L1A or GEO. You'll use the extracted L1A to generate an extracted GEO and L1B for processing through l2gen.

If you prefer to pass the extraction off to someone else...just use the L1/2 browser to request extracted L1A files for your region of interest.

Sean

Post Reply