Running simultaneous instances of multilevel_processor.py
Posted: Tue May 11, 2021 5:16 pm America/New_York
I am trying to run multilevel_processor as a job array on a cluster, to simultaneously run the code on multiple MODIS L1A files at once. I have followed the advice of past forum posts, here for example:
https://oceancolor.gsfc.nasa.gov/forum/ ... l?tid=5751
Where I have run get_anc.py and modis_atteph.py for each L1A file (in the file “create_anc_files.qsub”) prior to running multilevel_processor, which I do in the file “file_array.qsub”. In my parameter file “nir_settings.par” code I specify the --disable-download option and the --ancdir and --ancdb directories in the geo and l1a_extract_modis sections.
However, when I run this code for an L1A file directory with two files in it, only one of the files completes and the other fails at GEO (log files attached, and they have the naming convention of Seadas_NIR.o). For the file that was successful, an .anc file was created in the directory that I ran the code in, which makes me think that there is an error with my setup and the --disable download isn’t working as I am intending.
I have put all the files I have referred to here (as I wasn't able to attach them):
https://www.dropbox.com/sh/k79zgfxblqje ... FAoda?dl=0
Any advice would be appreciated.
Thanks,
Ted
https://oceancolor.gsfc.nasa.gov/forum/ ... l?tid=5751
Where I have run get_anc.py and modis_atteph.py for each L1A file (in the file “create_anc_files.qsub”) prior to running multilevel_processor, which I do in the file “file_array.qsub”. In my parameter file “nir_settings.par” code I specify the --disable-download option and the --ancdir and --ancdb directories in the geo and l1a_extract_modis sections.
However, when I run this code for an L1A file directory with two files in it, only one of the files completes and the other fails at GEO (log files attached, and they have the naming convention of Seadas_NIR.o). For the file that was successful, an .anc file was created in the directory that I ran the code in, which makes me think that there is an error with my setup and the --disable download isn’t working as I am intending.
I have put all the files I have referred to here (as I wasn't able to attach them):
https://www.dropbox.com/sh/k79zgfxblqje ... FAoda?dl=0
Any advice would be appreciated.
Thanks,
Ted