Page 1 of 1

query new files in directory

Posted: Thu Jul 28, 2022 4:34 pm America/New_York
by pcmehlitz
Hi,

I am periodically monitoring https://nrt3.modaps.eosdis.nasa.gov/archive/allData/5200/VJ114IMG_NRT/Recent to detect new entries in near realtime. Since this directory can get quite large I was wondering if the API (which obviously does support queries as in "...?fields=all&formats=csv") allows to filter entries newer than a provided date/time or at least allows to set the fields for the response ("...?fields=name.." did not work). I can work around it on my end but was wondering if there is a way to reduce bandwidth / load on the LANCE server.
Thanks,
Peter

Re: query new files in directory

Posted: Fri Jul 29, 2022 3:41 pm America/New_York
by LAADS_UserServices_M
Dear Peter:
One option is that you use:
https://nrt3.modaps.eosdis.nasa.gov/archive/allData/5200/VNP14IMG_NRT/2022/[latest DOY]
instead and monitor this folder for new files. DOY folders are updated at the same time as "Recent" folder.
Otherwise, you can use something like:
https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=VNP14IMG_NRT&temporalRanges=2022-07-26..2022-07-29

You can also explore LAADS API at:
https://ladsweb.modaps.eosdis.nasa.gov/tools-and-services/#api-v2
Replace:
"https://ladsweb.modaps.eosdis.nasa.gov" in the example
with
"https://nrt3.modaps.eosdis.nasa.gov"

Re: query new files in directory

Posted: Sun Jul 31, 2022 4:26 pm America/New_York
by captlen
I have noticed that on July 29 the data file names I obtain from subscription 616 and I can not extract Turbidity Tiff images using the scripts I have in octave and Matlab. Why were these changed? I may be in the wrong location for this question. My email address is captlen@offshoresatelliteservices.com Would appreciate any help. Thank you

Re: query new files in directory

Posted: Mon Aug 01, 2022 7:28 am America/New_York
by LAADS_UserServices_M
@@captlen Your question is not clear and seems like it does not belong to this question chain. Please, post your question again separately (as a new question) with more information about the product you have question about and location your are downloading your data from, etc.

Re: query new files in directory

Posted: Mon Aug 01, 2022 8:11 am America/New_York
by captlen
Prior to July 29,22 my 616 subscription produced Modis Aqua K490 such as the following for years: A2022201074500.L2_LAC_OC.NC Now on 7-28-22 files have changed to:
AQUA_MODIS.20220724T165001.L2.OC.NRT.nc. These files are unusable with my octave and Matlab scripts. Was this changed an error?

Re: query new files in directory

Posted: Mon Aug 01, 2022 1:58 pm America/New_York
by pcmehlitz
the OpenAPI spec was what I was looking for.

The latest DOY dir works but it does not seem to support query parameters (e.g. fields or temporalRanges), only full directory listings. Is it supposed to handle queries?

Maybe I'm doing something wrong but the 'details' temporalRanges queries work only partially. For instance, using

"https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=VJ114IMG_NRT&formats=csv&temporalRanges=2022-212T00:00..2022-213T23:59"

I get the entries for 2022-212 but none for 2022-213. Am I correct in that day/time in the query refer to acquisition day/time?

Thanks,
-- Peter

Re: query new files in directory

Posted: Mon Aug 01, 2022 3:59 pm America/New_York
by LAADS_UserServices_M
Yes, it is acquisition day:time.