I am trying to download IMERG Final half hourly precipitation data using wget pulling data from the GES DISC subsetter to get them in the form of netCDF files. Getting the data to download itself is not an issue, but I am doing it over almost the whole time scale which is a lot of data. Based off my math and how much I initially started to download, it would take hundreds of hours. I have tried to find ways to speed this up but have run into authorization issues and 401 errors so I was wondering if anyone has any suggestions on how to accomplish this? I am doing this on a research computing cluster and I can only cue a session for 48 hrs max.
https://disc.gsfc.nasa.gov/datasets/GPM_3IMERGHH_07/summary?keywords=%22IMERG%20final%22
Is there a way to download GPM_3IMERGHH faster?
-
GES DISC-EDL - ezamkoff
- Posts: 17
- Joined: Wed Jul 03, 2024 7:48 am America/New_York
Re: Is there a way to download GPM_3IMERGHH faster?
You may try to run at most two concurrent subset download jobs, but that is not
100% foolproof, i.e. there may be failed jobs if the host becomes too busy.
Since these data are in the Cloud, another option is to do your analysis in the
Cloud.
Lastly, full-sized IMERGHH data can be downloaded from the Cloud in many
concurrent sessions.
Regards,
GES DISC User Services
100% foolproof, i.e. there may be failed jobs if the host becomes too busy.
Since these data are in the Cloud, another option is to do your analysis in the
Cloud.
Lastly, full-sized IMERGHH data can be downloaded from the Cloud in many
concurrent sessions.
Regards,
GES DISC User Services