Downloading ASDC Data with Python 3

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
kmmrao
Posts: 14
Joined: Wed Dec 29, 2021 1:53 am America/New_York
Answers: 0

Re: Downloading ASDC Data with Python 3

by kmmrao » Thu Dec 30, 2021 6:46 am America/New_York

Dear members,
I tried to download the batch data using the multiple files script of python. But it is taking a lot of time. One hour to download two files of 800MB on the fastest internet. Please help me in downloading bulk data.

Tags:

njester
Posts: 20
Joined: Sat Mar 06, 2021 9:03 am America/New_York
Answers: 0
Been thanked: 3 times

Re: Downloading ASDC Data with Python 3

by njester » Thu Dec 30, 2021 9:01 am America/New_York


kmmrao
Posts: 14
Joined: Wed Dec 29, 2021 1:53 am America/New_York
Answers: 0

Re: Downloading ASDC Data with Python 3

by kmmrao » Fri Jan 07, 2022 2:39 am America/New_York

Wget is not working on Windows10, wget is treated as a virus in Windows 10. So, I never tried.

njester
Posts: 20
Joined: Sat Mar 06, 2021 9:03 am America/New_York
Answers: 0
Been thanked: 3 times

Re: Downloading ASDC Data with Python 3

by njester » Mon Jan 10, 2022 12:57 pm America/New_York

What data are you trying to get?

vasu_007
Posts: 12
Joined: Tue Jun 21, 2022 3:15 am America/New_York
Answers: 0

Re: Downloading ASDC Data with Python 3

by vasu_007 » Wed Jun 29, 2022 12:13 pm America/New_York

I AM GETTING THE error listed below. what I have to do


ConnectionError: HTTPSConnectionPool(host='l5ftl01.larc.nasa.gov', port=443): Max retries exceeded with url: /ops/misrl2l3/MISR/MIL2ASAE.003/2021.06.01/MISR_AM1_AS_AEROSOL_P092_O114121_F13_0023.nc (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000007B55A43970>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

njester
Posts: 20
Joined: Sat Mar 06, 2021 9:03 am America/New_York
Answers: 0
Been thanked: 3 times

Re: Downloading ASDC Data with Python 3

by njester » Wed Jun 29, 2022 12:32 pm America/New_York

The provided example script is only meant to get data from links found in asdc.larc.nasa.gov/data/. It looks like your looking for MISR data, you should be able to get it form here using the script example above:
https://asdc.larc.nasa.gov/data/MISR/

vasu_007
Posts: 12
Joined: Tue Jun 21, 2022 3:15 am America/New_York
Answers: 0

Re: Downloading ASDC Data with Python 3

by vasu_007 » Wed Jun 29, 2022 12:50 pm America/New_York

YES njester
i know that, and i have requested the data from that only. i currently downloading one file using the URL but it is giving the above mentioned error. for your information same script is downloading data without a error on my other system

vasu_007
Posts: 12
Joined: Tue Jun 21, 2022 3:15 am America/New_York
Answers: 0

Re: Downloading ASDC Data with Python 3

by vasu_007 » Wed Jun 29, 2022 2:55 pm America/New_York

LOOK!!! it is giving error after downloading some files. what should I do:
Attachments
the error which is coming after downloading some files. why is so...please help
the error which is coming after downloading some files. why is so...please help
q11.JPG (36.19 KiB) Not viewed yet

vasu_007
Posts: 12
Joined: Tue Jun 21, 2022 3:15 am America/New_York
Answers: 0

Re: Downloading ASDC Data with Python 3

by vasu_007 » Thu Jun 30, 2022 4:41 am America/New_York

Please help me!!!..... it is urgent

njester
Posts: 20
Joined: Sat Mar 06, 2021 9:03 am America/New_York
Answers: 0
Been thanked: 3 times

Re: Downloading ASDC Data with Python 3

by njester » Thu Jun 30, 2022 9:44 am America/New_York

Hello vasu_007
I'd love to investigate the python issue your seeing more, but since you need the your data fast, it might be better to try wget instead. If your able to use wget, I'd suggest trying this guide:
viewtopic.php?t=2329

Could you send me the top level URL of the data you want to download, what system your on, and what version of Python your using so that I can try to reproduce the issue?

Post Reply