"Too many requests" error when using OPeNDAP
Posted: Sun Mar 06, 2022 11:54 am America/New_York
I am starting to get "Too many requests" errors when using OPeNDAP to retrieve ocean color data. I am trying to utilize OPeNDAP rather than download large amounts of data to my computer, and am making many requests to get, for example, Rrs at a given wavelength from a single point across many days. For example (this code won't run because variables aren't defined, but you get the idea):
>>> for day in days:
>>> print(Dataset(filename(day)).variable['Rrs_412'][latind,lonind])
where an example of filename(day) is: [link] ://oceandata.sci.gsfc.nasa.gov/opendap/SeaWiFS/L3SMI/2000/001/S2000001.L3m_DAY_RRS_Rrs_412_9km.nc
for, say, a full year. I'll get "Too many requests" errors and then won't be able to access certain files. Is this approach not kosher? I.e., should I be downloading data and not trying to use the cloud for this? I don't recall this happening when I was doing things like this years ago, but maybe something changed and this is no longer a good way to deal with OceanColor data.
The specific error I am given is:
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <html^><head><title>429 Too Many Requests</title></head><body><center><h1>429 Too Many Requests</h1></center><hr><center>nginx</center></body></html>
>>> for day in days:
>>> print(Dataset(filename(day)).variable['Rrs_412'][latind,lonind])
where an example of filename(day) is: [link] ://oceandata.sci.gsfc.nasa.gov/opendap/SeaWiFS/L3SMI/2000/001/S2000001.L3m_DAY_RRS_Rrs_412_9km.nc
for, say, a full year. I'll get "Too many requests" errors and then won't be able to access certain files. Is this approach not kosher? I.e., should I be downloading data and not trying to use the cloud for this? I don't recall this happening when I was doing things like this years ago, but maybe something changed and this is no longer a good way to deal with OceanColor data.
The specific error I am given is:
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <html^><head><title>429 Too Many Requests</title></head><body><center><h1>429 Too Many Requests</h1></center><hr><center>nginx</center></body></html>