l3mapgen is extremely slow when generating 500 m daily mosaic over China region – any optimization suggestions?

[a1241637592]

I am currently processing MODIS Aqua data using SeaDAS/OCSSW with the following workflow:

Download MODIS L1A data covering China region for a given day

Generate Level-2 products using l2gen

Perform regional binning using l2bin (China spatial subset)

Use l3mapgen to generate a Level-3 mapped daily mosaic over China and resample to 500 m resolution

The main issue is that the l3mapgen step is extremely slow, even when running on a relatively high-performance server.

Server configuration:

CPU: 32 vCPU Intel Xeon Platinum 8352V @ 2.10 GHz

Memory: 60 GB RAM

OS: Linux (command-line OCSSW environment)

My goal is to generate a regional daily mosaic (China only, not global) in netCDF4 format.
The projection is Plate Carrée and interpolation method is nearest neighbor.

During the l2bin step, the spatial domain is already clipped to the China bounding box to avoid creating a global L3 bin file.
The same spatial bounds are also explicitly specified in the l3mapgen step.

Key processing parameters:

l2bin:

resolution = 500 m

prodtype = regional

bounding box: 73°E–135°E, 18°N–54°N

l3mapgen:

resolution = 500 m

projection = platecarree

interp = nearest

oformat = netcdf4

same bounding box as above

Despite these spatial constraints, l3mapgen still shows significant performance bottlenecks.

I would like to ask:

Are there recommended ways to optimize l3mapgen performance (e.g., parameter tuning, memory settings, projection choices)?

For regional high-resolution (500 m) daily mosaics, is there a more efficient recommended workflow?

Are there alternative approaches (for example, reprojecting L3 bin data using external tools) that could significantly improve performance?

Below is my current processing script (Python wrapper calling OCSSW command-line tools):
[attach code]

Any advice or best practices would be greatly appreciated.

[OB SeaDAS - dshea]

At 500m you might be running into a cache size problem with l3mapgen writing NetCDF.
Try running l3mapgen with "deflate=0". The file will be a lot bigger without compression, but this should speed up processing if the cache is the problem. You can always run h5repack on the file to compress it.

don