Forum

Sean,I am an individual running on the cluster. No one else is currently using it for OCSSW processing. I had changed my script (yesterday) to only make two calls per PDS with 3 seconds between PDS files in the loop.So for example PDS file MOD00.P2004086.1640_1.PDS.bz2  results in these 2 calls...

To avoid processing problems in a clustered computing environment, I am trying to download all the anc files in advance. I run modis_atteph.py and getanc.py with the no-download option first. When it returns, I parse it and check to see if the files exist. If they do not exist, the command is run ag...

BTW,Christopher told me that I should try the curl or wget retries, but I D/L all the sat data before processing, and these failures come from your script calls. My failures started up again just now:getanc.py /work/b/bmurch/tmp/S4P_MODIS_H5_MODIST.BERMUDA.2004.276_version_0.05_e1s6aoLH/MOD021KM.200...

Paul,
Thanks, it from a clustered environment and they all use xx.xxx.xx.xx as the outbound ipaddy. Is this the range you checked?
Brock

Tommy,
I had sent them a note, haven't heard back. Some are getting though, others not.
Cheers,
Brock

Tommy, Would this have affected:getanc.py MYD021KM.2019085.1515.S_BRAZIL.hdf --ancdb=./ancillary_data.dbError! could not establish a network connection. Check your network connection.If you do not find a problem, please try again laterANDmodis_GEO.py MOD00.P2003324.0720.CBAY.L1A_LAC -o MOD00.P200332...

$ curl --interface 2607:fe50:0:6330::115 --output /cms_zfs/sat_data/modis/l0/2003/151/MOD00.P2003151.0035_1.PDS.bz2 --retry 5 --retry-delay 2 --max-time 0 --remote-name-all https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MOD00.P2003151.0035_1.PDS.bz2  % Total    % Received % Xferd...

Sean,

I have sorted through all the problem files and reprocessed them from the start. All succeeded.
It must have been the transient file system problem that caused the error.
Sorry to have wasted your time.

Cheers,
Brock

Sean,

Version seadas-7.5.
I found a file system problem that may have affected this. I am going to try again, and see if the problems persist.

I'll let you know.
Brock

Sean,Here is the PAR file contents:cat /shares/cms_optics/bmurch/holding/S4P_MODIS_H5_MODIST.C_ATLANTIC.2002.243_version_0.05_i3QkMLCV/T20022431505.1KM.C_ATLANTIC.L2_LAC_OC.parifile=/work/b/bmurch/tmp/S4P_MODIS_H5_MODIST.C_ATLANTIC.2002.243_version_0.05_i3QkMLCV/MOD021KM.2002243.1505.C_ATLANTIC.hdfg...