l2gen failed for specific PACE/OCI L1B file

[OB WebDev - norman]

Hi,

I'm not sure if this is a question for Don, Tommy, or someone else.

I was processing a set of 34 PACE/OCI L1B files with l2gen [msl12 9.11.0-V2025.1 (Jan 30 2025 23:47:26)] today. All of them except one processed successfully. The file that failed was this one.

http://oceandata.sci.gsfc.nasa.gov/getfile/PACE_OCI.20250227T011247.L1B.V3.nc

My command line was this.

Code: Select all

l2gen ifile=PACE_OCI.20250227T011247.L1B.V3.nc ofile=PACE_OCI.20250227T011247.L2.nc l2prod=rhos_432,rhos_435,rhos_437,rhos_440,rhos_442,rhos_445,rhos_447,rhos_450,rhos_452,rhos_455,rhos_545,rhos_547,rhos_550,rhos_553,rhos_555,rhos_558,rhos_560,rhos_563,rhos_565,rhos_568,rhos_665,rhos_666,rhos_667,rhos_668,rhos_670,rhos_671,rhos_672,rhos_673,rhos_675,rhos_676

All of the L1B files I have downloaded are a few bytes larger than the size listed at
https://oceandata.sci.gsfc.nasa.gov/directdataaccess/Level-1B/PACE-OCI/2025/27-Feb-2025/
but that doesn't seem to cause an issue for the other files.

The failed scene also spans the antimeridian, but so do other adjacent scenes that processed successfully.

The error message returned was

Code: Select all

line 632 of /home/dshea/ocssw/oel_hdf4/libl1/l1_oci.c: NetCDF: HDF error

.

The full l2gen output is attached.

Any ideas as to what the problem might be?

Thanks,
Norman

error.pdf

(174.87 KiB) Downloaded 2314 times

[OB.DAAC - SeanBailey]

Hi Norman,

Not sure. I was able to process that file with no issues. It died for you reading the red focal plane data (that's what is at line 632 of libl1/l1_oci.c ...well, line 631...632 is the read error trap) for a scan line somewhere between lines 50 and 100...a very odd thing to do since it read previous lines just fine. Did you run out of disk space? Maybe RAM? (running OCI through l2gen eats up about 4G).
I hate to give the "works for me" response, but I'm grasping at straws.

BTW, if you're just outputting rhos, you can set atmocor=0 to speed up the processing a smidgen (no need for the cycles generating the aerosol reflectance since if its not used )

Sean

[OB WebDev - norman]

Hi Sean,

Hmm. It's after 9pm on a Friday. I was assuming that someone would maybe look at this sometime on Monday.

Sorry to be a bother. It's not a disk space issue since my script successfully processed several more L1B files to L2. Memory could be a possibility. I do sometimes push my Linux box pretty hard in that department. I'll try running it again tomorrow and keep an eye on my resources.

Thanks for the atmocor=0 tip. I think I used to do that but had forgotten.

I'm betting that you're still leading in forum responses.

I hope that you and family (and colleagues) are doing OK in this break-everything era we are living in. I'm pleased to see that NASA data are still available for now.

Norman

[OB WebDev - norman]

I tried processing the L1B file again (with atmocor=0 this time) and kept an eye on memory usage while it was running. My linux box never approached a memory limitation, but l2gen failed with the same error message. Since this was one failure out of many successful runs, and since Sean kindly provided the L2 file from his test run, I'm going to chalk this up to some unique configuration issue on my end and not pursue it further.

Thanks, Sean, for your help.

Norman