How to optimize processing many L1 files into GeoTIFFS (l3mapgen)?
Posted: Wed May 25, 2022 11:09 am America/New_York
Hello,
I'm preparing to process several years of MODIS Aqua L1 data into high resolution (250m) true color GeoTIFF images for a visualization/mapping project. Before I process the whole dataset I want to make sure that I've optimized each step.
Here's how I'm currently generating an image:
modis_GEO A2021137180500.L1A_LAC.x.hdf
modis_L1B A2021137180500.L1A_LAC.x.hdf A2021137180500.GEO.x.hdf
l2gen ifile=A2021137180500.L1B_LAC.x.hdf geofile=A2021137180500.GEO.x.hdf ofile=l2gen_out.nc l2prod=rhos_nnn north=45.9 south=40.5 east=-63.0 west=-71.2 resolution=250
l2bin ifile=l2gen_out.nc ofile=l2bin_out.nc prodtype=regional latnorth=46 latsouth=40 lonwest=-72 loneast=-63 resolution=Q
l3mapgen ifile=l2bin_out.nc ofile=truecolor.tiff oformat=tiff product=rgb use_transparency=yes interp=bin resolution=250.0 projection=mercator trimNSEW=1 north=45.9 south=40.5 east=-63.0 west=-71.2
I've already taken a couple steps to optimize:
- Requested "extracted" data when ordering the data from the portal
- Using crop/trim options where available (l2gen, l2bin, l3mapgen)
Knowing that the end goal is to generate high resolution, cloud-free images, are there any parameters to the commands that I'm using that will speed things up?
Is there a tool or API that can estimate cloud cover from L1 (or pre-generated L3 products) such that I could skip processing days that are too cloudy? It seems like cloud cover data isn't available until after the l2gen step (time consuming).
Once I get my dataset and parameters figured out I'll wrap this into a multiprocessing python script.
Thanks for any help.
I'm preparing to process several years of MODIS Aqua L1 data into high resolution (250m) true color GeoTIFF images for a visualization/mapping project. Before I process the whole dataset I want to make sure that I've optimized each step.
Here's how I'm currently generating an image:
modis_GEO A2021137180500.L1A_LAC.x.hdf
modis_L1B A2021137180500.L1A_LAC.x.hdf A2021137180500.GEO.x.hdf
l2gen ifile=A2021137180500.L1B_LAC.x.hdf geofile=A2021137180500.GEO.x.hdf ofile=l2gen_out.nc l2prod=rhos_nnn north=45.9 south=40.5 east=-63.0 west=-71.2 resolution=250
l2bin ifile=l2gen_out.nc ofile=l2bin_out.nc prodtype=regional latnorth=46 latsouth=40 lonwest=-72 loneast=-63 resolution=Q
l3mapgen ifile=l2bin_out.nc ofile=truecolor.tiff oformat=tiff product=rgb use_transparency=yes interp=bin resolution=250.0 projection=mercator trimNSEW=1 north=45.9 south=40.5 east=-63.0 west=-71.2
I've already taken a couple steps to optimize:
- Requested "extracted" data when ordering the data from the portal
- Using crop/trim options where available (l2gen, l2bin, l3mapgen)
Knowing that the end goal is to generate high resolution, cloud-free images, are there any parameters to the commands that I'm using that will speed things up?
Is there a tool or API that can estimate cloud cover from L1 (or pre-generated L3 products) such that I could skip processing days that are too cloudy? It seems like cloud cover data isn't available until after the l2gen step (time consuming).
Once I get my dataset and parameters figured out I'll wrap this into a multiprocessing python script.
Thanks for any help.