Size of 1 day level 2 data on global scale

Please enter here to ask a question about any NASA Science related topics!
Post Reply
emaure
Posts: 24
Joined: Tue Mar 26, 2013 7:18 am America/New_York

Size of 1 day level 2 data on global scale

by emaure » Thu Mar 19, 2020 5:18 am America/New_York

I am sorry to ask this question, but I found no better place to ask.
I would very much appreciate if anyone could help me get a sense of the estimate of 1 day level 2 data size globally.
I ask this because I am interested in looking at the coastal regions globally using the best available pixel resolution, that is 1km for SeaWiFS | Aqua | MERIS and 750 m for VIIRS.
NASA provides level 3 bin files but these are 4 km and in the very nearshore regions this resolution is limiting.
I appreciate if I could get a figure just to help me have a sense of how much it would take to work with the coastal water pixels globally.

Thank you for your help.
Sincerely,
Eligio

Tags:

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 136
Joined: Tue Feb 09, 2021 8:19 am America/New_York

Size of 1 day level 2 data on global scale

by OB WebDev - norman » Thu Mar 19, 2020 8:57 am America/New_York

Hi Eligio,

You can get daily level-2 volumes by adding up the file sizes in bytes
from pages like this representative Aqua/MODIS page from yesterday.

https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/L2/2020/078/

If you are only interested in coastal regions, you will obviously have
less data to process.  You should be aware, though, that different
coastal regions "look" different from each other because of differing
local ecologies, atmospheric conditions, amounts of sun glint, etc.
The level-2 products that we distribute are tuned to give the best
results globally at the expense of local (and especially coastal) accuracy.

Regards,
Norman

emaure
Posts: 24
Joined: Tue Mar 26, 2013 7:18 am America/New_York

Size of 1 day level 2 data on global scale

by emaure » Tue Mar 24, 2020 5:22 am America/New_York

Dear Norman,
Many thanks for the helpful information.
I just have a small thing to clarify .

MERIS data is in '*.bz2' compressed format.
By checking a few days I get the following result
MERIS
=====
  Year: 2007 | Day: 123 | Date: 2007-05-03 00:00:00 | TotalSize (GiB): 1.1766
  Year: 2006 | Day: 003 | Date: 2006-01-03 00:00:00 | TotalSize (GiB): 1.2714
  Year: 2009 | Day: 301 | Date: 2009-10-28 00:00:00 | TotalSize (GiB): 0.8674


My understanding is that, the TotalSize is for compressed file.
So, I downloaded one file, decompressed and compared it with the compressed one
From that, I get the size difference between the two as shown below.

compressed: 67.8547
decompressed: 620.6037
ratio (decompressed/compressed): 9.1461


The question is, can I safely assume that the compression ratio is about 9x for all files?
Thank you
Eligio

OB WebDev - norman
Subject Matter Expert
Subject Matter Expert
Posts: 136
Joined: Tue Feb 09, 2021 8:19 am America/New_York

Size of 1 day level 2 data on global scale

by OB WebDev - norman » Tue Mar 24, 2020 1:45 pm America/New_York

Hi Eligio,

Compression ratio will vary depending on the contents of the scene.
L2 scenes that have large areas masked out (e.g. for land) will
compress much smaller than those with a more complete and
varying feature set.

If I were you, I would set up a processing flow that downloads
scenes serially and determines their disposition before moving on
to the next scene.  In this way, you can only retain the data that
are useful for your purpose from each scene.

Norman

Post Reply