We are investigating, but the problem is not consistent so diagnosing it is proving a challenge. Stay tuned.
Sean
File Search broken ?
-
- User Services
- Posts: 1290
- Joined: Wed Sep 18, 2019 6:15 pm America/New_York
- Been thanked: 1 time
Re: File Search broken ?
Thanks a lot Sean!
I have done more analysis:
I have done more analysis:
- previously I split my search requests by sensor and month
- I tried to reduce the split period to 10 days but I still had timeouts, see the detailed log below: most of the time the requests are fast (note that our code ensure a minimum delay of 5s between each HTTP request to your API), but some cases are much more longer (25s then 60s in the log), seems like a lag in your API?
- then I tried to reduce the split period to 5 days, it seems working better with an homogeneous requests duration and I haven't any timeout, so for the moment I will use this workaround
Code: Select all
2022-06-30 10:52:04,327 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-02-16', 'edate': '2022-02-26', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:52:09,331 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-02-27', 'edate': '2022-03-09', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:52:14,334 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-03-10', 'edate': '2022-03-20', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:52:39,912 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-03-21', 'edate': '2022-03-31', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:53:39,198 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-04-01', 'edate': '2022-04-11', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:53:44,202 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-04-12', 'edate': '2022-04-22', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
...
2022-06-30 10:55:54,319 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'aqua', 'sdate': '2022-02-16', 'edate': '2022-02-26', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'A*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:55:59,323 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'aqua', 'sdate': '2022-02-27', 'edate': '2022-03-09', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'A*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 10:56:04,327 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'aqua', 'sdate': '2022-03-10', 'edate': '2022-03-20', 'psdate': '2021-12-12', 'pedate': None, 'dtype': 'L2', 'search': 'A*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-06-30 11:01:04,430 [ERROR] requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='oceandata.sci.gsfc.nasa.gov', port=443): Read timed out. (read timeout=300)
Re: File Search broken ?
Hi,
The issue seems getting worse: since 2022-07-05 4pm UTC my workaround to split by 5 days no more works and at least this request always timeouts (after 300s):
I also see that the other successful requests are longer than usual, often more than 20 seconds, example:
I tried to reduce the search period to only two days then one day, it was better but I still get timeouts! Example:
I don't know if it helps but the timeouts seem happen for Aqua, where Terra requests have no issues.
For the moment I was able to avoid timeouts by using one day and search products only for the past 20 days.
Julien
The issue seems getting worse: since 2022-07-05 4pm UTC my workaround to split by 5 days no more works and at least this request always timeouts (after 300s):
Code: Select all
2022-07-06 08:38:49,753 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'aqua', 'sdate': '2021-12-18', 'edate': '2021-12-23', 'psdate': '2021-12-18', 'pedate': None, 'dtype': 'L2', 'search': 'A*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
Code: Select all
2022-07-06 08:06:01,766 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2021-12-30', 'edate': '2022-01-04', 'psdate': '2021-12-18', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-07-06 08:06:27,798 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-01-05', 'edate': '2022-01-10', 'psdate': '2021-12-18', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-07-06 08:06:53,273 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-01-11', 'edate': '2022-01-16', 'psdate': '2021-12-18', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
2022-07-06 08:07:18,678 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'terra', 'sdate': '2022-01-17', 'edate': '2022-01-22', 'psdate': '2021-12-18', 'pedate': None, 'dtype': 'L2', 'search': 'T*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
Code: Select all
2022-07-06 11:05:21,209 [INFO] session.request('POST', 'https://oceandata.sci.gsfc.nasa.gov/api/file_search', data={'sensor': 'aqua', 'sdate': '2021-12-18', 'edate': '2021-12-19', 'psdate': '2021-12-18', 'pedate': None, 'dtype': 'L2', 'search': 'A*.L2_LAC_OC.nc', 'std_only': 1, 'cksum': 1, 'results_as_file': '1', 'format': 'txt'})
For the moment I was able to avoid timeouts by using one day and search products only for the past 20 days.
Julien