modis_geo.py processing problem

Please enter here to ask a question about any NASA Science related topics!
Post Reply
alakes
Posts: 23
Joined: Fri Aug 26, 2016 7:46 am America/New_York

modis_geo.py processing problem

by alakes » Thu Jan 16, 2020 2:08 am America/New_York

Hi,
My processing chain was working fine till yesterday. I have updated the data download process using ~/.netrc as mentioned in https://oceancolor.gsfc.nasa.gov/data/download_methods/ and it is working fine.

However, modis_geo.py fails  to work both in GUI & in command line.

My command for creating GEO file was

$SeaDASPath/scripts/modis_GEO.py $L1A_File -o $GEOFile --verbose --enable-dem --threshold=95

which returns error as follows
_____________________________________________________________________________________________
Determining required attitude and ephemeris files...
Searching database: /home/o-mc/SeaDAS7.5/ocssw/var/ancillary_data.db
()
Input file: A2020015054000.L1A_LAC
Sensor    : modisa
Start time: 2020015054000
End time  : 2020015054500
()
Downloading 'PM1EPHND_NRT.A2020015.0545.061' to /home/o-mc/SeaDAS7.5/ocssw/var/anc/2020/015
Connection interrupted, retrying up to 5 more time(s)
Connection error, retrying up to 4 more time(s)
Connection interrupted, retrying up to 3 more time(s)
Connection interrupted, retrying up to 2 more time(s)
Connection interrupted, retrying up to 1 more time(s)
URL attempted: oceandata.sci.gsfc.nasa.gov
Well, this is embarrassing...an error occurred that we just cannot get past...
Here is what we know: ('The read operation timed out',)
Please retry this request at a later time.
*** ERROR: The HTTP transfer failed with status code 302.
*** Please check your network connection and for the existence of the remote file:
*** oceandata.sci.gsfc.nasa.gov/cgi/getfile/PM1EPHND_NRT.A2020015.0545.061
***
*** Also check to make sure you have write permissions under the directory:
*** /home/o-mc/SeaDAS7.5/ocssw/var/anc/2020/015
()
Cannot create geolocation from A2020015054000.L1A_LAC; exiting.
_____________________________________________________________________________________________

I guess somebody can enlighten me how to fix this problem..

Thank you in advance..

Tags:

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1171
Joined: Wed Sep 18, 2019 6:15 pm America/New_York

modis_geo.py processing problem

by OB.DAAC - SeanBailey » Fri Jan 17, 2020 7:08 am America/New_York


pavel_babyak
Posts: 12
Joined: Wed Sep 04, 2019 10:46 am America/New_York

modis_geo.py processing problem

by pavel_babyak » Mon Jan 20, 2020 1:20 am America/New_York

Some workaround of problem with sites
This patch use .netrc, so you should add to .netrc not only `urs.earthdata.nasa.gov` but `oceandata.sci.gsfc.nasa.gov` too

patch

diff --git a/modules/ProcUtils.py b/modules/ProcUtils.py
index edceb41..547fe7b 100644
--- a/modules/ProcUtils.py
+++ b/modules/ProcUtils.py
@@ -6,7 +6,9 @@ SeaDAS library for commonly used functions within other python scripts
from __future__ import print_function

import sys
+import netrc

+netrc_dict=netrc.netrc()

#  ------------------ DANGER -------------------
#
@@ -98,6 +100,7 @@ def _httpdl(url, request, localpath='.', outputfilename=None, ntries=5,
         verbose - get chatty about connection issues (boolean, default False)
     """
     global ofile
+    global netrc_dict
     import os
     import re
     import socket
@@ -114,6 +117,13 @@ def _httpdl(url, request, localpath='.', outputfilename=None, ntries=5,
         os.umask(0o02)
         os.makedirs(localpath, mode=0o2775)

+    cred=netrc_dict.authenticators(url)
+    if cred is not None:
+        from base64 import b64encode
+        ( _user, _acc, _pass ) = cred
+        userAndPass = b64encode(bytes(_user + ':' + _pass, "utf-8"))
+        reqHeaders.update({ 'Authorization' : 'Basic %s' %  userAndPass })
+
     urlConn, proxy = httpinit(url, timeout=timeout, urlConn=urlConn)

     try:


IMHO this is better soluthion than this https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=11520

josl
Posts: 2
Joined: Wed Jan 22, 2020 1:33 pm America/New_York

modis_geo.py processing problem

by josl » Tue Jan 28, 2020 7:02 am America/New_York

This is obsolete since the latest update to the scripts, right? https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=11598

OB SeaDAS - dshea
Subject Matter Expert
Subject Matter Expert
Posts: 214
Joined: Thu Mar 05, 2009 10:25 am America/New_York

modis_geo.py processing problem

by OB SeaDAS - dshea » Tue Jan 28, 2020 11:04 am America/New_York

Correct.  The latest update fixes this issue.

don

Post Reply