Download subset of VNP02IMG data.

[b.coerver]

Hello,

I'm using VNP02IMG data for a project, and more specifically only the following three subdatasets:

- /observation_data/I05_quality_flags
- /observation_data/I05
- /observation_data/I05_brightness_temperature_lut

Until recently, I could download subsets of this dataset through S3, but apparently this was a loophole which is now closed (see here for more info: https://github.com/nsidc/earthaccess/discussions/489).

So I think there are two other options left for me to access this data (https://cmr.earthdata.nasa.gov/search/concepts/C2105091501-LAADS.html):

(1) Using a direct download (https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/5200/VNP02IMG/)
(2) Using OpenDAP (https://ladsweb.modaps.eosdis.nasa.gov/opendap/RemoteResources/laads/allData/5200/VNP02IMG/contents.html)

As far as I know it's impossible to download a subset of the data through option (1). For example when I run:

gdalinfo "/vsicurl/https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/5200/VNP02IMG/2016/015/VNP02IMG.A2016015.0030.002.2021082161921.nc"

I get a message saying "ERROR 1: Range downloading not supported by this server!"

So my last resort is option (2), unfortunately, none of the subdatasets that are included in the netcdf files are listed in the opendap. I guess opendap doesnt work with subdatasets? Or is this a bug? Or am I doing something wrong?

Is it really only possible to work with this data by downloading the entire file (from which I only need about 20%)?

Thanks a lot for any help or insights!

Regards,
Bert

[LAADS_UserServices_M]

Dear Bert:
You can use LAADS post processing tool to perform spatial and/or parameter subset on files you need. This is available at the last step (REVIEW & ORDER) of the ordering process using LAADS DAAC's "Find Data" interface at:
https://ladsweb.modaps.eosdis.nasa.gov/search/order/5/VNP02IMG--5200/

This way you can order only the parameters (SDS) you need. If this is the first time you are using post processing tool, you can consult “How to Use LAADS DAAC Post-Processing Tools” tutorial available at:
https://ladsweb.modaps.eosdis.nasa.gov/learn

For OPeNDAP, you can consult various examples provided at:
https://ladsweb.modaps.eosdis.nasa.gov/tools-and-services#opendap

[b.coerver]

Hi,

Thanks for your reply.

- About the Post-Processing tool: I saw that, but I don't think it's possible to access this through an API? Also there would be queues and waiting times involved, which is not ideal.

- About openDAP: I've seen the examples and used openDAP before. But I don't think those examples apply here (perhaps because the data is stored inside subdatasets?). If you go to the link here (https://ladsweb.modaps.eosdis.nasa.gov/opendap/RemoteResources/laads/allData/5200/VNP02IMG/2018/009/VNP02IMG.A2018009.0036.002.2021078103914.nc.dmr.html) for example, there is nothing listed under "variables" and "global dimensions". Also when I click on the "Get As Netcdf4" download button, I get a ~11kb file that only contains the global metadata but not the actual data. (even though in the file listing here [https://ladsweb.modaps.eosdis.nasa.gov/opendap/RemoteResources/laads/allData/5200/VNP02IMG/2018/009/contents.html], its says the file has a size of 271169961 bytes).

So if someone knows how to access those subdatasets through opendap, i'd be very happy to know!

Bert

[LAADS_UserServices_M]

For OPeNDAP, you must be following discussion with Nathan Potter on your ticket to support-laads.
So, while the OPeNDAP issue is worked on and get resolved you can explore LAADS "LWS Classic" API option. This is available at:
https://ladsweb.modaps.eosdis.nasa.gov/tools-and-services/#lwsclassic

Here under "LWS Classic" / "Quick Start Guide" you can find information on how to write script to search and order files with examples of perl scripts?

[b.coerver]

Hello,

The OPeNDAP issue is resolved. OPeNDAP now works for LAADS datasets that have variables stored inside "groups" (I tested a few of them).

Thanks a lot for your help!

Regards,
Bert