Downloading ASDC Data with Python 3
Re: Downloading ASDC Data with Python 3
Dear members,
I tried to download the batch data using the multiple files script of python. But it is taking a lot of time. One hour to download two files of 800MB on the fastest internet. Please help me in downloading bulk data.
I tried to download the batch data using the multiple files script of python. But it is taking a lot of time. One hour to download two files of 800MB on the fastest internet. Please help me in downloading bulk data.
Tags:
Re: Downloading ASDC Data with Python 3
You could see wget is faster?
viewtopic.php?f=7&t=2329&sid=a3d86dfa41 ... e4f8fab95e
viewtopic.php?f=7&t=2329&sid=a3d86dfa41 ... e4f8fab95e
Re: Downloading ASDC Data with Python 3
Wget is not working on Windows10, wget is treated as a virus in Windows 10. So, I never tried.
Re: Downloading ASDC Data with Python 3
I AM GETTING THE error listed below. what I have to do
ConnectionError: HTTPSConnectionPool(host='l5ftl01.larc.nasa.gov', port=443): Max retries exceeded with url: /ops/misrl2l3/MISR/MIL2ASAE.003/2021.06.01/MISR_AM1_AS_AEROSOL_P092_O114121_F13_0023.nc (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000007B55A43970>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))
ConnectionError: HTTPSConnectionPool(host='l5ftl01.larc.nasa.gov', port=443): Max retries exceeded with url: /ops/misrl2l3/MISR/MIL2ASAE.003/2021.06.01/MISR_AM1_AS_AEROSOL_P092_O114121_F13_0023.nc (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000007B55A43970>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))
Re: Downloading ASDC Data with Python 3
The provided example script is only meant to get data from links found in asdc.larc.nasa.gov/data/. It looks like your looking for MISR data, you should be able to get it form here using the script example above:
https://asdc.larc.nasa.gov/data/MISR/
https://asdc.larc.nasa.gov/data/MISR/
Re: Downloading ASDC Data with Python 3
YES njester
i know that, and i have requested the data from that only. i currently downloading one file using the URL but it is giving the above mentioned error. for your information same script is downloading data without a error on my other system
i know that, and i have requested the data from that only. i currently downloading one file using the URL but it is giving the above mentioned error. for your information same script is downloading data without a error on my other system
Re: Downloading ASDC Data with Python 3
LOOK!!! it is giving error after downloading some files. what should I do:
- Attachments
-
- the error which is coming after downloading some files. why is so...please help
- q11.JPG (36.19 KiB) Not viewed yet
Re: Downloading ASDC Data with Python 3
Hello vasu_007
I'd love to investigate the python issue your seeing more, but since you need the your data fast, it might be better to try wget instead. If your able to use wget, I'd suggest trying this guide:
viewtopic.php?t=2329
Could you send me the top level URL of the data you want to download, what system your on, and what version of Python your using so that I can try to reproduce the issue?
I'd love to investigate the python issue your seeing more, but since you need the your data fast, it might be better to try wget instead. If your able to use wget, I'd suggest trying this guide:
viewtopic.php?t=2329
Could you send me the top level URL of the data you want to download, what system your on, and what version of Python your using so that I can try to reproduce the issue?