Satellite validation matchup tool val_extract issues

[cmitchell]

Hi,
I have been trying out the new matchup scripts provided in the ocssw source code: fd_matchup.py and mk_matchup.py. The fd_matchup.py script is running well, but I am having problems with mk_matchup.py script: it fails when trying to run val_extract.  Below is a stripped back example showing what I get when I run the command (note in my seabass file I have only one station, which does match in space and time with the provided VIIRS file).

cmitchell@ubuntu:/mnt/hgfs/SHARE/VIIRS_matchups_2017$ mk_matchup.py --seabass_file=VIIRSMatchupStations_test2.sb --sat_file='/mnt/hgfs/SHARE/VIIRS_matchups_2017/sat_files/V2013233164200.L2_SNPP_OC.nc'
sat_file = /mnt/hgfs/SHARE/VIIRS_matchups_2017/sat_files/V2013233164200.L2_SNPP_OC.nc

Calculating satellite match-up for L2 file:
Unknown option: global_att

usage: mk_matchup.py [-h] --sat_file SAT_FILE --seabass_file SEABASS_FILE
                     [--out_file OUT_FILE] [--clobber [CLOBBER]] [--slat SLAT]
                     [--elat ELAT] [--slon SLON] [--elon ELON]
                     [--box_size BOX_SIZE]
                     [--min_valid_sat_pix MIN_VALID_SAT_PIX]
                     [--max_time_diff MAX_TIME_DIFF]
                     [--max_coeff_variation MAX_COEFF_VARIATION]
mk_matchup.py: error: val_extract failed -- ensure that the val_extract binary is compiled and on your PATH and that /mnt/hgfs/SHARE/VIIRS_matchups_2017/sat_files/V2013233164200.L2_SNPP_OC.nc exists

Any help would be much appreciated. Thanks.

[jpsco]

Hi,

Thank you for checking out the new stand-alone match-up tools, and I apologize that you are getting an error using mk_matchup.py.

From the mk_matchup.py output you are providing, it sounds like your copy of SeaDAS and its OCSSW components may be an outdated version of the val_extract binary that is used to extract the satellite data from your VIIRS file. To confirm if that is the case, would you please run on your command-line:

> val_extract version=1

and let us know what the output is. It should look something like this:

val_extract=2.6.2
val_extract-main=1.3.1

Cheers,
Joel

[cmitchell]

Thanks for the quick response.

Looks like that may be it. I'm getting

val_extract=2.5.9
val_extract-main=1.2.2

Cheers,
Catherine

[jpsco]

Hi Catherine,

I believe that version difference is the source of the problem.

However, I double-checked and val_extract version 2.5.9 is, in fact, the latest publicly available version of val_extract, which is due to a bug on our end.
We will push an updated version of the val_extract binaries to the SeaDAS / OCSSW code repository shortly (probably early next week sometime).

Once the version is updated on our end, you will then be able to update your OCSSW source code installation from the OCSSW menu in SeaDAS, and that should resolve the problem.

Thank you for your patience while we resolve this!

Cheers,
Joel

[cmitchell]

OK great, thanks a lot Joel.

Please let me know when the updated version is available.

[cmitchell]

Hi Joel,

Is there any progress on this?

Many thanks,
Catherine

[OB.DAAC - SeanBailey]

Catherine,
The fix is not as easy as Joel believed it would be. ...well, it's not hard, but it is more involved than a simple script update and requires attention that we don't have the time to give just yet.  Hopefully, things will settle down soon, but for now there are too many hamster wheels, not enough hamsters...

Sean

[jpsco]

Hi,
The val_extract binary has been updated in the latest version of SeaDAS via its copy of OCSSW. To get the mk_matchup.py script to work, you will need to update your local val_extract binary to the latest public version, which should be: val_extract=2.6.2 and val_extract-main=1.3.1.
To do so, please open SeaDAS and from the "OCSSW" drop-down menu, select "Update OC Processors" be sure that at least one box is checked (e.g. - "Source Code, " etc) and click "Run." Upon successful execution of the update, your val_extract version should be compatible with the mk_matchup.py script, enabling the script's full functionality. Please post again on the forum should you have any additional trouble with the stand-alone match-up tools.

Cheers,
Joel

[cmitchell]

Hi,

Thanks for letting me know about the update. I updated through the GUI as you described, but am still getting the same error message as I originally did when I then try to run the mk_matchup.py script. I ran

val_extract version=1

and am still getting val_extract=2.5.9 val_extract-main=1.2.2. I tried updating at the command line (running the install_ocssw.py script with the --src option) but that doesn't work either. I should maybe clarify: everything appears to update OK, but when checking the val_extract binary, it still seems to be the older version.

Thanks
Catherine

[jpsco]

Hi Catherine,
I'm sorry that you are still experiencing difficulties with val_extract. Both methods that you described to update OCSSW in SeaDAS (GUI and command-line) are valid. Are you by any chance running on a Mac operating system instead of Linux?

It appears that the Linux binaries have been updated:~/seadas-7.4/ocssw/bin$ ./val_extract version=1
val_extract=2.6.2
val_extract-main=1.3.1

Yet, I believe the updated Mac build release is lagging a bit behind the Linux binary version release. My apologies for the confusion.

Cheers,
Joel