Equation of calculating statistics parameter in level-3 bin data

Please enter here to ask a question about any NASA Science related topics!
Post Reply
hzhao
Posts: 5
Joined: Tue Jun 23, 2020 12:41 pm America/New_York

Equation of calculating statistics parameter in level-3 bin data

by hzhao » Wed Jun 24, 2020 5:12 pm America/New_York

Hello,

Recently, I’m trying to get chlorophyll values from level-3 bin products.

According to SeaWiFS Technical Report Vol. 32, the equation used to calculate the mean value of variable is:

mx = S1 / w
square(Sx) = S2 / w – square(mx)
Xmle = exp( mx + ½ * square(Sx) );

I assume that S1 is sum and S2 is sum_square . When I applied this equation, the results seem not right (e.g. maximum value can reach 10e11, all values are above 1). In contrast, If I remove exp, the results are more reasonable but which is not mentioned in NASA report. Could you give me some instructions about this question. Are other variables such as PIC being applied a log calculation as well?

Thanks,
Haipeng

Tags:

gnwiii
Posts: 605
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 1

Equation of calculating statistics parameter in level-3 bin data

by gnwiii » Wed Jun 24, 2020 6:02 pm America/New_York

Is there a reason you can't use the OCSSW software, in particular, l3bindump (for ASCII output) and l3mapgen (for images)?  We have been promised improvements in the binning capabilities for the next version of the OCSSW software.   The sources for the OCSSW programs are available.  Others have implemented calculations with binned data in Matlab, Python, and IDL.

If you are writing your own software, note that calculations of the number of bins in each latitude band can give slightly different numbers if the latitudes are not computed exactly as done in the NASA software.  This is more of a problem when the bins are small.

hzhao
Posts: 5
Joined: Tue Jun 23, 2020 12:41 pm America/New_York

Equation of calculating statistics parameter in level-3 bin data

by hzhao » Wed Jun 24, 2020 10:43 pm America/New_York

Hi George,

Thanks for your reply. I am used to using MATLAB and C to process datasets. In my case, I just wrote a script in MATLAB to calculate parameters including lat/lon and chlor_a for each bin. I have not installed virtual machine in my windows and no experience of using the OCSSW. I'm not sure whether the OCSSW software is flexible to be used on a time series study. Whatever, I would have a look at those sources. Hope to find my answer.

Thanks,
Haipeng

gnwiii
Posts: 605
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 1

Equation of calculating statistics parameter in level-3 bin data

by gnwiii » Thu Jun 25, 2020 10:42 am America/New_York

With Windows 10 Pro it is easy (and no cost) to enable WSL2 and install Ubuntu 18.04 (seems to be the most widely used for OCSSW processing) from the Windows "Store".  You want WSL2 as it uses VM technology, but if your Windows is a locked down "corporate standard" image then WSL and WSL2 may not be available.
WSL gives you a bash shell in a terminal, which is all you need to install and run the OCSSW software.   You will find it helpful to verify your Matlab results against the OCSSW software using l3bindump or l3mapgen.  The source code is of course the up-to-date documentation.   In Windows you could also just download the source as a git bundle as described in OCSSW.    Once you have unpacked the bundle do a "git pull" to get any changes.  

With minor tweaks the NASA IDL scripts should run under GNU Data Language (GDL) for Windows.

I have used l3bindump on (linux and macOS) many times to create ASCII files for a list of bins or region-of-interest for time-series processing. Linux (or macOS) is generally faster than Windows when running loops in scripts.   I only use WSL2 for one-off calculations where speed isn't critical as it is much slower than native linux.

hzhao
Posts: 5
Joined: Tue Jun 23, 2020 12:41 pm America/New_York

Equation of calculating statistics parameter in level-3 bin data

by hzhao » Thu Jun 25, 2020 4:25 pm America/New_York

Very helpful info! I once installed SeaBASS in Windows and followed an instruction (https://seadas.gsfc.nasa.gov/client_server/) to set up WSL2 but couldn't make OCSSW run correctly. Now, I know that maybe I should install the software directly into WSL2; doing the calculations in a Linux system still seems the best choice though.

Thanks for sharing your experience.

Haipeng

gnwiii
Posts: 605
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 1

Equation of calculating statistics parameter in level-3 bin data

by gnwiii » Thu Jun 25, 2020 6:58 pm America/New_York

The OCSSW software is generally very efficient, but I/O bound.  An older PC that isn't up to running Windows 10 can be very useful for OCSSW batch processing with linux if you don't need l2gen.  The best return on investment is to install a solid-state-disk (SSD).

Post Reply