Page 1 of 1

Size of 1 day level 2 data on global scale

Posted: Thu Mar 19, 2020 5:18 am America/New_York
by emaure
I am sorry to ask this question, but I found no better place to ask.
I would very much appreciate if anyone could help me get a sense of the estimate of 1 day level 2 data size globally.
I ask this because I am interested in looking at the coastal regions globally using the best available pixel resolution, that is 1km for SeaWiFS | Aqua | MERIS and 750 m for VIIRS.
NASA provides level 3 bin files but these are 4 km and in the very nearshore regions this resolution is limiting.
I appreciate if I could get a figure just to help me have a sense of how much it would take to work with the coastal water pixels globally.

Thank you for your help.

Size of 1 day level 2 data on global scale

Posted: Thu Mar 19, 2020 8:57 am America/New_York
by OB WebDev - norman
Hi Eligio,You can get daily level-2 volumes by adding up the file sizes in bytesfrom pages like this representative Aqua/MODIS page from yesterday. (/MODIS-Aqua/L2/2020/078/). If you are only interested in coastal regions, you will obviously haveless data to process.  You should be aware, though, that differentcoastal regions "look" different from each other because of differinglocal ecologies, atmospheric conditions, amounts of sun glint, etc.The level-2 products that we distribute are tuned to give the bestresults globally at the expense of local (and especially coastal) accuracy.Regards,Norman

Size of 1 day level 2 data on global scale

Posted: Tue Mar 24, 2020 5:22 am America/New_York
by emaure
Dear Norman,
Many thanks for the helpful information.
I just have a small thing to clarify .

MERIS data is in '*.bz2' compressed format.
By checking a few days I get the following result
  Year: 2007 | Day: 123 | Date: 2007-05-03 00:00:00 | TotalSize (GiB): 1.1766
  Year: 2006 | Day: 003 | Date: 2006-01-03 00:00:00 | TotalSize (GiB): 1.2714
  Year: 2009 | Day: 301 | Date: 2009-10-28 00:00:00 | TotalSize (GiB): 0.8674

My understanding is that, the TotalSize is for compressed file.
So, I downloaded one file, decompressed and compared it with the compressed one
From that, I get the size difference between the two as shown below.

compressed: 67.8547
decompressed: 620.6037
ratio (decompressed/compressed): 9.1461

The question is, can I safely assume that the compression ratio is about 9x for all files?
Thank you

Size of 1 day level 2 data on global scale

Posted: Tue Mar 24, 2020 1:45 pm America/New_York
by OB WebDev - norman
Hi Eligio,

Compression ratio will vary depending on the contents of the scene.
L2 scenes that have large areas masked out (e.g. for land) will
compress much smaller than those with a more complete and
varying feature set.

If I were you, I would set up a processing flow that downloads
scenes serially and determines their disposition before moving on
to the next scene.  In this way, you can only retain the data that
are useful for your purpose from each scene.