Page 1 of 1

Missing data between L1&2 Browser and Order Data lists

Posted: Fri Jun 19, 2020 6:18 pm America/New_York
by madjidh
Hello,

I downloaded a list from "order data" in L1&2 browser, for my area (20°E-13°W | 50°N-70°N) which contains all the MODIS A since the beginning.
However, I was suspicious about one image that was very small; so I digged a bit.
It turns out that "Order data" and the manual way to download an image doesn't give me the same files.

For example, if I pick the 10th of Dec, 2016, with order data, I have :
A2016345142000.L1A_LAC
A2016345124500.L1A_LAC

However, when I use the browser, I have
A2016345124000.L1A_LAC, which is a bit different from the above (5 min)
A2016345124500.L1A_LAC. I know it's the end of the image, but I don't know why such a behavior ?
(I also have the 14:20:00 data but not relevant here).
However, that one is missing from my "order data list" and is within my coordinates :
A2016345110500.L1A_LAC

Browser link where the 12:45:00 data isn't shown :
https://oceancolor.gsfc.nasa.gov/cgi/browse.pl?sub=level1or2list&per=DAY&day=17145&prm=CHL&set=10&ndx=2&mon=17136&sen=amod&rad=1200&frc=0&n=56.6015625&w=-3.8671875&wid=170&hgt=254&id=1592603848.17198&cn=67.94&cs=44.60&cw=-25.86&ce=18.24&dnm=D

The missing data at 12:45:00 is in the middle of my area while for the 11:05:00 one just a few pixels should be available.
Do you have an explanation for this ? Did I miss an option with the Order Data ?

Best

Missing data between L1&2 Browser and Order Data lists

Posted: Fri Jun 19, 2020 7:10 pm America/New_York
by OB ODPS - jgwilding
I'm wondering if this could be a day-night flag issue.  I noticed that the last argument in your URL is dnm=D which indicates daytime data.  The 1245 granule is listed as a mixed (M) granule so it might be getting excluded.

I assume that in your first example for order data, you swapped the '2' and '4' in the first file name, so it should be A2016345124000.L1A_LAC?  That is listed as a daytime (D) granule.

Normally selecting 'Day' should select all granules with any daytime data, which will include those with mixed, and that is also true when you select 'Night'.  There might be a query that doesn't have that implemented correctly.

You can play around with your query URL, changing the dnm=D to dnm=M to see if that gets the 1245 granule to show up.

john

Missing data between L1&2 Browser and Order Data lists

Posted: Sat Jun 20, 2020 7:44 pm America/New_York
by madjidh
Hi John

I assume that in your first example for order data, you swapped the '2' and '4' in the first file name, so it should be A2016345124000.L1A_LAC?  That is listed as a daytime (D) granule.

-> No, I really have these two files, one at 14:20 on the west part of my area (Ireland), and two at 12:40/12:45 with the 45th one being the north part (Scotland/Iceland) of the 12:40 image, not present within the 12:40 file.

I just work with daytime data and I didn't specified a night mode in order data. Changing the URL doesn't make the 12:45 appear. It's really a difference between the data available with "Order Data" and with the L1&2 Browser GUI/click method.

Missing data between L1&2 Browser and Order Data lists

Posted: Mon Jun 22, 2020 11:19 am America/New_York
by OB WebDev - norman
Hi all,

The missing scene in the browser search results is caused by a logic bug in my browse.pl
code that effects mixed day/night granules that span the prime meridian.  Tommy pointed
this one out to me some years ago (I still have a note pinned to my bulletin board at
Goddard should we ever get back there), but I've given it lower priority since the plan
is to replace the current browse code with new logic anyway.

Norman

Missing data between L1&2 Browser and Order Data lists

Posted: Mon Jun 22, 2020 5:53 pm America/New_York
by madjidh
Hi Norman, thanks for the answer it makes sense now.

I just ordered the data again and now I see every data that should be present I have 6 files for the 10-12-2016, while I had only 2 in the past.
However I am a bit concerned about why I didn't get the same results when I ordered the same data a couple of month ago.

I think that I did a mistake when I turned the file list copy pasted from the browser into a text file as I had 15 000 files, and now have 60 000 files (and there are 4 columns :eek: ).

I hope that I'm the only one who did this mistake, maybe adding a "download list of files" button could help for the future browser / logic ?

Anyway thanks for the answer
Regards