TooManyRedirects

[user_gis]

Good day!

We have a problem with TERRA/AQUA processing.
When we try downloads ATT/EPH data, using modis_atteph.py utility  we get back error message:

OBPG session started
Downloading 'AM1EPHN0.A2020175.0800.002' to /home/proc/OCSSW/var/anc/2020/175
Traceback (most recent call last):
File "/home/proc/OCSSW/scripts/modis_GEO.py", line 194, in <module> sys.exit(main())
File "/home/proc/OCSSW/scripts/modis_GEO.py", line 185, in main m.atteph()
File "/home/proc/OCSSW/scripts/modules/modis_GEO_utils.py", line 240, in atteph get.locate()
File "/home/proc/OCSSW/scripts/modules/anc_utils.py", line 616, in locate verbose=self.verbose)
File "/home/proc/OCSSW/scripts/modules/ProcUtils.py", line 73, in httpdl with obpgSession.get(urlStr, stream=True, timeout=timeout) as req:
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 546, in get return self.request('GET', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 668, in send history = [resp for resp in gen] if allow_redirects else []
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 165, in resolve_redirects
raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, response=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

Whats wrong here?

Regards,
Dmitry

[OB.DAAC - SeanBailey]

Dmitry,

Sorry for the delay in responding...
Are you still having issues?  We recently identified a condition for users that did not have an "Organization" set in their Earthdata Login profile that could result in this redirect problem.
We've removed the requirement as it is unnecessary for most downloads (required only for ESA Sentinel and MERIS data).
If you still have issues, please add "-vv" to your modis_atteph.py (or modis_GEO.py) calls and capture the verbose output, save to a log and attached to a post so we can better diagnose your problem. 

Regards,
Sean

[mcanto]

Hi Sean,

I had the same "Too many redirects" issue when running modis_GEO.py last week. It went away when I reset the .netrc (to include both urs.earthdata.nasa.gov and oceancolor.gsfc.nasa.gov) and .urs_cookies following instruction at https://oceancolor.gsfc.nasa.gov/data/download_methods/.
Did that really resolve it? or I got lucky? :confused:
And your suggestion to add "-vv" should be the norm from now and we don't have to bother setting the .netrc and .urs_cookies for getting MODIS data? Sorry for the naive question!

Thanks,
Tess

[OB.DAAC - SeanBailey]

Tess,

Not sure.  There is nothing on our side that requires a line in the .netrc for oceancolor.gsfc.nasa.gov, but it does seem to fix some issues for folks (as the client might need it).
This problem I mentioned was a very particular case, and we've resolved that on our side (couldn't hurt to add a organization to your Earthdata Login profile...just in case :grin:)
If you use the python scripts provided by identify and retrieve data from our servers e.g. modis_atteph.py, modis_GEO.py (which calls modis_atteph), obdaac_download; then you do not need the .urs_cookies file.  That is only needed for cURL and wget (the name of the file could be anything, as you have to tell the client what it is).  The .netrc file *is* used by all, so you'll definitely need that.
...but may not need an entry for oceandata, but maybe it does... fun, eh?

Sean