opendap issues (I think)

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
btupper
Posts: 27
Joined: Fri Nov 17, 2006 4:14 pm America/New_York
Answers: 0

opendap issues (I think)

by btupper » Fri Feb 03, 2023 11:30 am America/New_York

Hello,

I am having unexpected trouble working with some opendap resources. On one hand, I can't seem to get a connection that isn't malformed - see the output below. On the other hand, I seem to be able to request and receive the data I want. It's a puzzler!

Should I be worried?

Cheers,
Ben

Code: Select all

netcdf library version 4.8.1 of Apr  1 2022 13:10:44 $
btupper@ecocast ~ $ ncdump -h https://oceandata.sci.gsfc.nasa.gov/opendap/MODISA/L3SMI/2022/0728/AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km.nc
syntax error, unexpected WORD_WORD, expecting ';' or ','
context: Attributes { poc { String long_name "Particulate Organic Carbon, D. Stramski, 2007 (443/555 version)"; Float32 scale_factor 0.200000003; Float32 add_offset 6400.00000; String units "mg m^-3"; Int16 _FillValue -32767; Int16 valid_min -32000; Int16 valid_max -27000; String reference "Stramski, D., et al. "Relationships^ between the surface concentration of particulate organic carbon and optical properties in the eastern South Pacific and eastern Atlantic Oceans." Biogeosciences 5.1 (2008): 171-201."; String display_scale "log"; Float32 display_min 10.0000000; Float32 display_max 1000.00000; } lat { String long_name "Latitude"; String units "degrees_north"; String standard_name "latitude"; Float32 _FillValue -999.000000; Float32 valid_min -90.0000000; Float32 valid_max 90.0000000; } lon { String long_name "Longitude"; String units "degrees_east"; String standard_name "longitude"; Float32 _FillValue -999.000000; Float32 valid_min -180.000000; Float32 valid_max 180.000000; } palette { } NC_GLOBAL { String product_name "AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km.nc"; String instrument "MODIS"; String title "MODISA Level-3 Standard Mapped Image"; String project "Ocean Biology Processing Group (NASA/GSFC/OBPG)"; String platform "Aqua"; String source "satellite observations from MODIS-Aqua"; String temporal_range "day"; String processing_version "R2022.0"; String date_created "2022-11-23T19:24:41.000Z"; String history "l3mapgen par=AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km.nc.param "; String l2_flag_names "ATMFAIL,LAND,HILT,HISATZEN,STRAYLIGHT,CLDICE,COCCOLITH,LOWLW,CHLWARN,CHLFAIL,NAVWARN,MAXAERITER,ATMWARN,HISOLZEN,NAVFAIL,FILTER,HIGLINT"; String time_coverage_start "2022-07-28T00:00:01.000Z"; String time_coverage_end "2022-07-29T02:34:59.000Z"; Int32 start_orbit_number 107614; Int32 end_orbit_number 107630; String map_projection "Equidistant Cylindrical"; String latitude_units "degrees_north"; String longitude_units "degrees_east"; Float32 northernmost_latitude 90.0000000; Float32 southernmost_latitude -90.0000000; Float32 westernmost_longitude -180.000000; Float32 easternmost_longitude 180.000000; Float32 geospatial_lat_max 90.0000000; Float32 geospatial_lat_min -90.0000000; Float32 geospatial_lon_max 180.000000; Float32 geospatial_lon_min -180.000000; Float32 latitude_step 0.0416666679; Float32 longitude_step 0.0416666679; Float32 sw_point_latitude -89.9791641; Float32 sw_point_longitude -179.979172; String spatialResolution "4.638312 km"; String geospatial_lon_resolution "4.638312 km"; String geospatial_lat_resolution "4.638312 km"; String geospatial_lat_units "degrees_north"; String geospatial_lon_units "degrees_east"; Int32 number_of_lines 4320; Int32 number_of_columns 8640; String measure "Mean"; Float32 suggested_image_scaling_minimum 10.0000000; Float32 suggested_image_scaling_maximum 1000.00000; String suggested_image_scaling_type "LOG"; String suggested_image_scaling_applied "No"; String _lastModified "2022-11-23T19:24:41.000Z"; String Conventions "CF-1.6 ACDD-1.3"; String institution "NASA Goddard Space Flight Center, Ocean Ecology Laboratory, Ocean Biology Processing Group"; String standard_name_vocabulary "CF Standard Name Table v36"; String naming_authority "gov.nasa.gsfc.sci.oceandata"; String id "R2022.0/L3/AQUA_MODIS.20220728.L3b.DAY.POC.nc"; String license "https://science.nasa.gov/earth-science/earth-science-data/data-information-policy/"; String creator_name "NASA/GSFC/OBPG"; String publisher_name "NASA/GSFC/OBPG"; String creator_email "data@oceancolor.gsfc.nasa.gov"; String publisher_email "data@oceancolor.gsfc.nasa.gov"; String creator_url "https://oceandata.sci.gsfc.nasa.gov"; String publisher_url "https://oceandata.sci.gsfc.nasa.gov"; String processing_level "L3 Mapped"; String cdm_data_type "grid"; String identifier_product_doi_authority "http://dx.doi.org"; String identifier_product_doi "10.5067/AQUA/MODIS/L3M/POC/2022"; String keywords "Earth Science > Oceans > Ocean Chemistry > Organic Carbon"; String keywords_vocabulary "NASA Global Change Master Directory (GCMD) Science Keywords"; String data_bins "Attribute edlided: Unsupported attribute type (NC_INT64)"; Float32 data_minimum 10.3999023; Float32 data_maximum 12953.4014; }}
Illegal attribute
context: Attributes { poc { String long_name "Particulate Organic Carbon, D. Stramski, 2007 (443/555 version)"; Float32 scale_factor 0.200000003; Float32 add_offset 6400.00000; String units "mg m^-3"; Int16 _FillValue -32767; Int16 valid_min -32000; Int16 valid_max -27000; String reference "Stramski, D., et al. "Relationships^ between the surface concentration of particulate organic carbon and optical properties in the eastern South Pacific and eastern Atlantic Oceans." Biogeosciences 5.1 (2008): 171-201."; String display_scale "log"; Float32 display_min 10.0000000; Float32 display_max 1000.00000; } lat { String long_name "Latitude"; String units "degrees_north"; String standard_name "latitude"; Float32 _FillValue -999.000000; Float32 valid_min -90.0000000; Float32 valid_max 90.0000000; } lon { String long_name "Longitude"; String units "degrees_east"; String standard_name "longitude"; Float32 _FillValue -999.000000; Float32 valid_min -180.000000; Float32 valid_max 180.000000; } palette { } NC_GLOBAL { String product_name "AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km.nc"; String instrument "MODIS"; String title "MODISA Level-3 Standard Mapped Image"; String project "Ocean Biology Processing Group (NASA/GSFC/OBPG)"; String platform "Aqua"; String source "satellite observations from MODIS-Aqua"; String temporal_range "day"; String processing_version "R2022.0"; String date_created "2022-11-23T19:24:41.000Z"; String history "l3mapgen par=AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km.nc.param "; String l2_flag_names "ATMFAIL,LAND,HILT,HISATZEN,STRAYLIGHT,CLDICE,COCCOLITH,LOWLW,CHLWARN,CHLFAIL,NAVWARN,MAXAERITER,ATMWARN,HISOLZEN,NAVFAIL,FILTER,HIGLINT"; String time_coverage_start "2022-07-28T00:00:01.000Z"; String time_coverage_end "2022-07-29T02:34:59.000Z"; Int32 start_orbit_number 107614; Int32 end_orbit_number 107630; String map_projection "Equidistant Cylindrical"; String latitude_units "degrees_north"; String longitude_units "degrees_east"; Float32 northernmost_latitude 90.0000000; Float32 southernmost_latitude -90.0000000; Float32 westernmost_longitude -180.000000; Float32 easternmost_longitude 180.000000; Float32 geospatial_lat_max 90.0000000; Float32 geospatial_lat_min -90.0000000; Float32 geospatial_lon_max 180.000000; Float32 geospatial_lon_min -180.000000; Float32 latitude_step 0.0416666679; Float32 longitude_step 0.0416666679; Float32 sw_point_latitude -89.9791641; Float32 sw_point_longitude -179.979172; String spatialResolution "4.638312 km"; String geospatial_lon_resolution "4.638312 km"; String geospatial_lat_resolution "4.638312 km"; String geospatial_lat_units "degrees_north"; String geospatial_lon_units "degrees_east"; Int32 number_of_lines 4320; Int32 number_of_columns 8640; String measure "Mean"; Float32 suggested_image_scaling_minimum 10.0000000; Float32 suggested_image_scaling_maximum 1000.00000; String suggested_image_scaling_type "LOG"; String suggested_image_scaling_applied "No"; String _lastModified "2022-11-23T19:24:41.000Z"; String Conventions "CF-1.6 ACDD-1.3"; String institution "NASA Goddard Space Flight Center, Ocean Ecology Laboratory, Ocean Biology Processing Group"; String standard_name_vocabulary "CF Standard Name Table v36"; String naming_authority "gov.nasa.gsfc.sci.oceandata"; String id "R2022.0/L3/AQUA_MODIS.20220728.L3b.DAY.POC.nc"; String license "https://science.nasa.gov/earth-science/earth-science-data/data-information-policy/"; String creator_name "NASA/GSFC/OBPG"; String publisher_name "NASA/GSFC/OBPG"; String creator_email "data@oceancolor.gsfc.nasa.gov"; String publisher_email "data@oceancolor.gsfc.nasa.gov"; String creator_url "https://oceandata.sci.gsfc.nasa.gov"; String publisher_url "https://oceandata.sci.gsfc.nasa.gov"; String processing_level "L3 Mapped"; String cdm_data_type "grid"; String identifier_product_doi_authority "http://dx.doi.org"; String identifier_product_doi "10.5067/AQUA/MODIS/L3M/POC/2022"; String keywords "Earth Science > Oceans > Ocean Chemistry > Organic Carbon"; String keywords_vocabulary "NASA Global Change Master Directory (GCMD) Science Keywords"; String data_bins "Attribute edlided: Unsupported attribute type (NC_INT64)"; Float32 data_minimum 10.3999023; Float32 data_maximum 12953.4014; }}
netcdf AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km {
dimensions:
	eightbitcolor = 256 ;
	lat = 4320 ;
	lon = 8640 ;
	rgb = 3 ;
variables:
	float lat(lat) ;
	float lon(lon) ;
	byte palette(rgb, eightbitcolor) ;
	short poc(lat, lon) ;
}

Tags:

OB.DAAC - amscott
User Services
User Services
Posts: 345
Joined: Mon Jun 22, 2020 5:24 pm America/New_York
Answers: 1
Has thanked: 8 times
Been thanked: 3 times

Re: opendap issues (I think)

by OB.DAAC - amscott » Thu Feb 09, 2023 7:12 pm America/New_York

I haven't been able to replicate this issue. Let me pass it to our systems admins to see if anything is apparent to them. Have you tried surrounding the URL into quotation marks?

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 368
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

Re: opendap issues (I think)

by OB ODPS - towens » Fri Feb 10, 2023 7:45 am America/New_York

I can reproduce the error using netcdf library version 4.8.1 of Sept 29 2001 09:36:14

syntax error, unexpected WORD_WORD, expecting ';' or ','

I notice that when I run ncdump locally on the file I see:

ubyte palette(rgb, eightbitcolor) ;

But through OPeNDAP:
palette { }
with no terminating ";" and then the NC_GLOBAL{ starts

I think that is what it is complaining about.
Is the ubyte a problem for OPeNDAP?

Tommy

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

Re: opendap issues (I think)

by gnwiii » Mon Feb 13, 2023 10:18 am America/New_York

Using Fedora's netcdf-4.9.0, I see:

Code: Select all

String data_bins "Attribute edlided [SIC]: Unsupported attribute type (NC_INT64)";
using opendap.

Using the downloaded file:

Code: Select all

% ncdump -h AQUA_MODIS.20220728.L3m.DAY.POC.poc.4km.nc | grep -F LL
		:data_bins = 3699877LL ;
Maybe oceandata.sci.gsfc.nasa.gov/opendap doesn't support NC_INT64 --
https://docs.opendap.org/index.php?title=BES_-_Modules_-_The_NetCDF_Handler#:~:text=NC_INT64%2C%20NC_UINT64%20DAP2%20does%20not%20support%2064-bit%20integers,type%29%2C%20et%20c.%2C%20are%20called%20User%20Defined%20types. says:
DAP2 does not support 64-bit integers (this will be added soon to the next version of the protocol).

OB ODPS - towens
Subject Matter Expert
Subject Matter Expert
Posts: 368
Joined: Fri Feb 05, 2021 9:17 am America/New_York
Answers: 0
Been thanked: 4 times

Re: opendap issues (I think)

by OB ODPS - towens » Mon Feb 27, 2023 7:34 am America/New_York

We have updated our OPeNDAP server to the latest available version. Are you still seeing the error?

Tommy

gnwiii
Posts: 713
Joined: Fri Jan 29, 2021 5:51 pm America/New_York
Answers: 2
Has thanked: 1 time

Re: opendap issues (I think)

by gnwiii » Mon Feb 27, 2023 11:14 am America/New_York

I no longer get the many lines error report, but do still see (note the spelling change (edlided-->elided):

Code: Select all

:data_bins = "Attribute elided: Unsupported attribute type (NC_INT64)" ;
What is "data_bins"? Is it ever used with mapped data?

Post Reply