Hi all, for my thesis I am trying to download aerosol data from MERRA2. I would really like this data, since it goes back further than CAM4. However, I am working on a business-controlled laptop and cannot download any program required to retrieve the dataset files from the document with file links. I have never used wget before and got really confused.
Is there someone who can help me out by downloading it for me? I have the list of links ready-to-go.
MERRA2 aerosol data unable to download - who can do it for me?
-
GES DISC - xpan2
- Subject Matter Expert
- Posts: 39
- Joined: Mon Aug 16, 2021 2:49 pm America/New_York
Re: MERRA2 aerosol data unable to download - who can do it for me?
Dear User,
Given your constraints and that you already know which collection and variables you're interested in, we recommend using a Python script to download the data instead of wget.
Please refer to these two tutorials and choose the one that best fits your situation:
1. How to Subset and Download Level 2, 3, and 4 GES DISC Data Using Python (https://disc.gsfc.nasa.gov/information/howto?title=How%20to%20Subset%20and%20Download%20Level%202,%203,%20and%204%20GES%20DISC%20Data%20Using%20Python)
2. How to Access GES DISC Data Using Python (https://disc.gsfc.nasa.gov/information/howto?keywords=python&title=How%20to%20Access%20GES%20DISC%20Data%20Using%20Python)
If we can be of any further assistance, please let us know.
Regards,
GES DISC User Services
Given your constraints and that you already know which collection and variables you're interested in, we recommend using a Python script to download the data instead of wget.
Please refer to these two tutorials and choose the one that best fits your situation:
1. How to Subset and Download Level 2, 3, and 4 GES DISC Data Using Python (https://disc.gsfc.nasa.gov/information/howto?title=How%20to%20Subset%20and%20Download%20Level%202,%203,%20and%204%20GES%20DISC%20Data%20Using%20Python)
2. How to Access GES DISC Data Using Python (https://disc.gsfc.nasa.gov/information/howto?keywords=python&title=How%20to%20Access%20GES%20DISC%20Data%20Using%20Python)
If we can be of any further assistance, please let us know.
Regards,
GES DISC User Services
Re: MERRA2 aerosol data unable to download - who can do it for me?
Thank you for the quick reply. I tried both options.
No. 1 gives the error message in the attached image. The problem is that I need a large amount of files. I need preferably daily aerosol data for the dust extinction AOT and organic carbon extinction AOT over the period 1981-01-01 to 2024-12-31. My spatial grid is West Africa.
The second option with search and download works, but takes ages and slows down my laptop to the point where it's unworkable. Also the files are huge (1GB each) since I can't select only two of the variables.
I don't understand why it has to be so difficult to access this data. With ERA5 and CAMS (for other variables) I had very little problems and the download was much more straightforward...
No. 1 gives the error message in the attached image. The problem is that I need a large amount of files. I need preferably daily aerosol data for the dust extinction AOT and organic carbon extinction AOT over the period 1981-01-01 to 2024-12-31. My spatial grid is West Africa.
The second option with search and download works, but takes ages and slows down my laptop to the point where it's unworkable. Also the files are huge (1GB each) since I can't select only two of the variables.
I don't understand why it has to be so difficult to access this data. With ERA5 and CAMS (for other variables) I had very little problems and the download was much more straightforward...
- Attachments
-
-
GES DISC - cbattisto
- Subject Matter Expert
- Posts: 33
- Joined: Wed Feb 16, 2022 4:38 pm America/New_York
Re: MERRA2 aerosol data unable to download - who can do it for me?
Hello,
The Harmony service is currently having interruptions, but in the meantime, could you please upload the notebook you were using to query Harmony? Before the service was interrupted, I was able to recreate your subset for 1 day and the returned file was about 1MB big. Additionally, based on the amount of data you are querying (nearly the entire collection), the Harmony service will split it into several jobs and you may experience interruptions since it can potentially take a very long time.
An alternate option is to bypass Harmony and call the OPeNDAP service directly, although you would need to find the array indices of your bounding box to get the spatial subset. Otherwise, you could download the two variables you want, which per granule is about 30 MB, and then use Xarray to perform spatial subsetting. Again, since you are subsetting over 16,000 granules, you may experience interruptions and should probably split the temporal length into smaller chunks. Here is an OPeNDAP notebook you can use to try this: https://disc.gsfc.nasa.gov/information/howto?title=How%20to%20Access%20M2T1NXSLV%20Cloud%20OPeNDAP%20Granules%20using%20Python
The Harmony service is currently having interruptions, but in the meantime, could you please upload the notebook you were using to query Harmony? Before the service was interrupted, I was able to recreate your subset for 1 day and the returned file was about 1MB big. Additionally, based on the amount of data you are querying (nearly the entire collection), the Harmony service will split it into several jobs and you may experience interruptions since it can potentially take a very long time.
An alternate option is to bypass Harmony and call the OPeNDAP service directly, although you would need to find the array indices of your bounding box to get the spatial subset. Otherwise, you could download the two variables you want, which per granule is about 30 MB, and then use Xarray to perform spatial subsetting. Again, since you are subsetting over 16,000 granules, you may experience interruptions and should probably split the temporal length into smaller chunks. Here is an OPeNDAP notebook you can use to try this: https://disc.gsfc.nasa.gov/information/howto?title=How%20to%20Access%20M2T1NXSLV%20Cloud%20OPeNDAP%20Granules%20using%20Python