"NetCDF: HDF error" when reading netcdf files downloaded from EarthData opendap

[jinqj]

Hello there,

I have problems in reading netcdf4 files downloaded from EarthData opendap through the wget command. For example:

wget https://opendap.earthdata.nasa.gov/collections/C2087132178-GES_DISC/granules/S5P_L2__CO_____HiR.2%253AS5P_RPRO_L2__CO_____20180430T001950_20180430T020120_02818_03_020400_20220901T170054.nc.dap.nc4?dap4.ce=/PRODUCT/carbonmonoxide_total_column_corrected%5B0:1:0%5D%5B0:1:2905%5D%5B0:1:214%5D

Then when I use ncview to open the file, the following error shows up:

netcdf_fi_get_data: error on nc_get_vara_float call
cdfid=65536 variable=PRODUCT/carbonmonoxide_total_column_corrected
start, count:
[0]: 0 1
[1]: 0 2906
[2]: 0 215
NetCDF: HDF error

However, I can successfully "ncview" the netcdf file downloaded using the same command but for different dataset (CO vs. HCHO).

wget https://opendap.earthdata.nasa.gov/collections/C1918210023-GES_DISC/granules/S5P_L2__HCHO___HiR.2%253AS5P_RPRO_L2__HCHO___20180507T013006_20180507T031136_02918_03_020401_20230115T155629.nc.dap.nc4?dap4.ce=/PRODUCT/formaldehyde_tropospheric_vertical_column%5B0:1:0%5D%5B0:1:3244%5D%5B0:1:449%5D

Surprisingly, reducing the size of the scanline dimension or ground_pixel dimension by 1 helps get rid of the reading error via ncview. For example:

wget https://opendap.earthdata.nasa.gov/collections/C2087132178-GES_DISC/granules/S5P_L2__CO_____HiR.2%253AS5P_RPRO_L2__CO_____20180430T001950_20180430T020120_02818_03_020400_20220901T170054.nc.dap.nc4?dap4.ce=/PRODUCT/carbonmonoxide_total_column_corrected%5B0:1:0%5D%5B0:1:2904%5D%5B0:1:214%5D

wget https://opendap.earthdata.nasa.gov/collections/C2087132178-GES_DISC/granules/S5P_L2__CO_____HiR.2%253AS5P_RPRO_L2__CO_____20180430T001950_20180430T020120_02818_03_020400_20220901T170054.nc.dap.nc4?dap4.ce=/PRODUCT/carbonmonoxide_total_column_corrected%5B0:1:0%5D%5B0:1:2905%5D%5B0:1:213%5D

The above two commands can download readable files. I also tried to download TROPOMI aerosol index, aerosol layer height, and other variables, the similar "NetCDF: HDF error" shows up.

The failure occurs only when I request the entire 2-D spatial array:

2906 × 215 = 624,790 values → FAIL

2906 × 214 = 621,884 values → WORK
2905 × 215 = 624,575 values → WORK

I am wondering if someone on the Forum could help resolve this downloading issue.

Thanks a lot!
Qinjian Jin

[GES DISC - alouise517]

Dear Qinjian,

Our team wanted to let you know that we are still investigating your inquiry and will reach out as soon as we have a response to your questions and the issues you are facing.

If you have any additional questions or require further assistance in the meantime, please don't hesitate to post them here.

Regards,
GES DISC User Services

[jinqj]

Thanks for your reply and help! Looking forward to a solution to this issue.

Best regards,
Qinjian JIn