query new files in directory

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
pcmehlitz
Posts: 2
Joined: Thu Jul 28, 2022 4:13 pm America/New_York
Answers: 0

query new files in directory

by pcmehlitz » Thu Jul 28, 2022 4:34 pm America/New_York

Hi,

I am periodically monitoring https://nrt3.modaps.eosdis.nasa.gov/archive/allData/5200/VJ114IMG_NRT/Recent to detect new entries in near realtime. Since this directory can get quite large I was wondering if the API (which obviously does support queries as in "...?fields=all&formats=csv") allows to filter entries newer than a provided date/time or at least allows to set the fields for the response ("...?fields=name.." did not work). I can work around it on my end but was wondering if there is a way to reduce bandwidth / load on the LANCE server.
Thanks,
Peter

Tags:

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: query new files in directory

by LAADS_UserServices_M » Fri Jul 29, 2022 3:41 pm America/New_York

Dear Peter:
One option is that you use:
https://nrt3.modaps.eosdis.nasa.gov/archive/allData/5200/VNP14IMG_NRT/2022/[latest DOY]
instead and monitor this folder for new files. DOY folders are updated at the same time as "Recent" folder.
Otherwise, you can use something like:
https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=VNP14IMG_NRT&temporalRanges=2022-07-26..2022-07-29

You can also explore LAADS API at:
https://ladsweb.modaps.eosdis.nasa.gov/tools-and-services/#api-v2
Replace:
"https://ladsweb.modaps.eosdis.nasa.gov" in the example
with
"https://nrt3.modaps.eosdis.nasa.gov"
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

captlen
Posts: 14
Joined: Mon Jul 18, 2022 11:31 am America/New_York
Answers: 0

Re: query new files in directory

by captlen » Sun Jul 31, 2022 4:26 pm America/New_York

I have noticed that on July 29 the data file names I obtain from subscription 616 and I can not extract Turbidity Tiff images using the scripts I have in octave and Matlab. Why were these changed? I may be in the wrong location for this question. My email address is captlen@offshoresatelliteservices.com Would appreciate any help. Thank you

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: query new files in directory

by LAADS_UserServices_M » Mon Aug 01, 2022 7:28 am America/New_York

@@captlen Your question is not clear and seems like it does not belong to this question chain. Please, post your question again separately (as a new question) with more information about the product you have question about and location your are downloading your data from, etc.
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

captlen
Posts: 14
Joined: Mon Jul 18, 2022 11:31 am America/New_York
Answers: 0

Re: query new files in directory

by captlen » Mon Aug 01, 2022 8:11 am America/New_York

Prior to July 29,22 my 616 subscription produced Modis Aqua K490 such as the following for years: A2022201074500.L2_LAC_OC.NC Now on 7-28-22 files have changed to:
AQUA_MODIS.20220724T165001.L2.OC.NRT.nc. These files are unusable with my octave and Matlab scripts. Was this changed an error?

pcmehlitz
Posts: 2
Joined: Thu Jul 28, 2022 4:13 pm America/New_York
Answers: 0

Re: query new files in directory

by pcmehlitz » Mon Aug 01, 2022 1:58 pm America/New_York

the OpenAPI spec was what I was looking for.

The latest DOY dir works but it does not seem to support query parameters (e.g. fields or temporalRanges), only full directory listings. Is it supposed to handle queries?

Maybe I'm doing something wrong but the 'details' temporalRanges queries work only partially. For instance, using

"https://nrt3.modaps.eosdis.nasa.gov/api/v2/content/details?products=VJ114IMG_NRT&formats=csv&temporalRanges=2022-212T00:00..2022-213T23:59"

I get the entries for 2022-212 but none for 2022-213. Am I correct in that day/time in the query refer to acquisition day/time?

Thanks,
-- Peter

LAADS_UserServices_M
User Services
User Services
Posts: 279
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 3 times

Re: query new files in directory

by LAADS_UserServices_M » Mon Aug 01, 2022 3:59 pm America/New_York

Yes, it is acquisition day:time.
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

Post Reply