Earthdata Forum

**Title:** MODIS `l2gen` is very slow at 500 m resolution from existing L1B+GEO files — any recommended parameters or workflow improvements?

Hello SeaDAS / OCSSW team,

I am processing **MODIS Aqua** data with **existing L1B and GEO files**, and generating L2 products at **500 m resolution** using `l2gen`.

My main issue is that **`l2gen` is extremely slow** at 500 m resolution, and each individual process uses only a small fraction of one CPU core. I would like to ask whether this is expected, whether there are any recommended parameters for improving performance, or whether there may be something wrong in my workflow.

## Environment

* Platform: Linux (AutoDL server)
* OCSSW root: `/root/autodl-fs/ocssw`
* Python: `/root/miniconda3/bin/python3`
* Processing mode: existing **L1B + GEO -> L2**
* Resolution: **500**
* Parallel strategy:

* `getanc.py` runs serially
* `l2gen` runs in parallel across many scenes

## What I observed

For one scene, `l2gen` processing is very slow.
Example progress output looks like this:

```text
Processing scan # 50 (51 of 4060) after 311 seconds
Processing scan # 100 (101 of 4060) after 577 seconds
Processing scan # 150 (151 of 4060) after 837 seconds
Processing scan # 200 (201 of 4060) after 1086 seconds
Processing scan # 250 (251 of 4060) after 1344 seconds
Processing scan # 300 (301 of 4060) after 1594 seconds
Processing scan # 350 (351 of 4060) after 1850 seconds
Processing scan # 400 (401 of 4060) after 2100 seconds
Processing scan # 450 (451 of 4060) after 2354 seconds
Processing scan # 500 (501 of 4060) after 2599 seconds
```

This is roughly about **5 seconds per scan**, so one full scene can take many hours.

At the same time:

* each `l2gen` process uses only about **20%–30% of one CPU core**
* total system CPU usage remains low
* disk I/O is also low
* memory is sufficient

So this does **not** appear to be a CPU saturation or disk bottleneck problem.

## What I already checked

I monitored the server with `top` and `iostat`.

### CPU observations

* CPU idle is often around **90%**
* iowait is near **0%**
* memory is sufficient

### I/O observations

Typical `iostat -x 1` output shows:

* `%util` is very low (roughly 0–3%)
* `await` is very low
* no obvious storage bottleneck

This suggests the main limitation is **inside `l2gen` itself**, or perhaps some aspect of 500 m processing for MODIS.

## Current command / parameters

I am using a Python pipeline that:

1. scans existing `L1B` and `GEO`
2. runs `getanc.py` serially
3. runs `l2gen` in parallel

The main `l2gen` call is conceptually like this:

```bash
l2gen \
ifile=<L1B file> \
geofile=<GEO file> \
ofile=<output L2 file> \
par=<anc file> \
l2prod=rhos_469,rhos_555,rhos_645,rhos_859,rhos_1240,l2_flags \
resolution=500 \
oformat=netcdf4 \
proc_ocean=1 \
maskland=0 \
deflate=0
```

## Questions

1. Is this level of slowness for **MODIS `l2gen` at 500 m** expected?
2. Are there any recommended parameters to improve performance for this case?
3. Are there known performance issues in recent OCSSW / `l2gen` versions for MODIS 500 m processing?
4. Is there a recommended workflow for better throughput besides launching many separate `l2gen` processes?
5. Does `resolution=500` require any additional considerations regarding HKM companion files that could affect performance?

## Code structure

My script is a stable pipeline with:

* serial `getanc.py`
* recovery of `.anc` from log if needed
* parallel `l2gen`
* automatic HKM link repair for 500 m

If helpful, I can also provide the full Python script.

Any advice would be greatly appreciated. Thank you very much.

What version of OCSSW are you running?
You might want to try installing the "--arch=odps" if your linux kernel is new enough to see if that helps.
When I run a MODIS granule at resolution=500, I get mostly CPU saturation.
If you are not CPU or IO limited and the memory is only about 1.2G, I don't know where to look for a bottle neck.

Here is my test setup.

My system is a decent Ubuntu 20.04 server
OCSSW V2026.0 arch=odps (Ubuntu 20.04 build)
(install_ocssw --arch=odps ...)

I ran the first 500 lines using this par file:

ifile=AQUA_MODIS.20251105T003500.L1B.hdf
geofile=AQUA_MODIS.20251105T003500.GEO.hdf
ofile=AQUA_MODIS.20251105T003500.L2.500.hdf
par=AQUA_MODIS.20251105T003500.L1B.hdf.anc
l2prod=rhos_469,rhos_555,rhos_645,rhos_859,rhos_1240,l2_flags
resolution=500
oformat=netcdf4
proc_ocean=1
maskland=0
deflate=0
eline=500

Output looks like this:

Processing scan # 50 (51 of 500) after 9 seconds
Processing scan # 100 (101 of 500) after 16 seconds
Processing scan # 150 (151 of 500) after 23 seconds
Processing scan # 200 (201 of 500) after 30 seconds
Processing scan # 250 (251 of 500) after 36 seconds
Processing scan # 300 (301 of 500) after 43 seconds
Processing scan # 350 (351 of 500) after 51 seconds
Processing scan # 400 (401 of 500) after 58 seconds
Processing scan # 450 (451 of 500) after 67 seconds
...
End MSl12 processing at 2026085141738000
Processing Rate = 6.666667 scans/sec

output if "time -v l2gen ..."

User time (seconds): 74.19
System time (seconds): 0.88
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:15.08
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 1221348
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 379189
Voluntary context switches: 193
Involuntary context switches: 7325
Swaps: 0
File system inputs: 2945298
File system outputs: 173470
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

Also note that my MacBook Pro 2023 is quite a bit faster than this Linux server running l2gen.

don

Hi Don,

Thank you for the detailed reply and the benchmark result.

I checked my environment and found that I am currently running:

* `l2gen 9.11.0-V2025.5`
* Ubuntu 22.04.3
* Linux kernel `5.15.0-86-generic`

So my setup is different from yours, since you tested with `OCSSW V2026.0 arch=odps`.

I am now going to install a separate `V2026.0` environment with `--arch=odps` and repeat the test with the same input style, including a short `eline=500` benchmark, to compare performance and CPU usage.

My current issue is that `l2gen` at 500 m is still extremely slow and does not come close to saturating one CPU core, so I agree it makes sense to first rule out a version / build difference.

I will update the thread after I finish the comparison.

Thanks again.