Downloading data from CDDIS archive.
Posted: Wed Mar 15, 2023 9:29 am America/New_York
My code
proxy ={"https": f"http://{username}:{password}@{proxyserver}:{port}"}
url = 'https://cddis.nasa.gov/archive/gnss/data/daily/2017/003/17o/mate0030.17o.Z'
response = requests.get(url, proxies=proxy) auth=(A_username, A_password))
if response.status_code == 200:
# Check the Content-Length header to get the expected file size
expected_size = int(response.headers.get("Content-Length", 0))
# Write the downloaded content to a file
with open('mate0030.17o.Z', 'wb') as f:
for chunk in response.iter_content(chunk_size=1000):
if chunk:
f.write(chunk)
print(f"Downloaded {len(chunk)} bytes...")
# Check the actual size of the downloaded file
actual_size = os.path.getsize('mate0030.17o.Z')
# Compare the expected and actual sizes to ensure the file was fully downloaded
if actual_size == expected_size:
print("File downloaded successfully!")
else:
print("File download incomplete.")
else:
print("Unable to download file.")
output of the code
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 178 bytes...
File download incomplete.
I have queries as listed below
1)I am able to access the file with my python code, but my data down is going incomplete. why?
2)How should the content of .netrc file should be feeded to my code? Here I am feeding the user credentials manually as I have no idea of interfacing this details.
Please suggest.
proxy ={"https": f"http://{username}:{password}@{proxyserver}:{port}"}
url = 'https://cddis.nasa.gov/archive/gnss/data/daily/2017/003/17o/mate0030.17o.Z'
response = requests.get(url, proxies=proxy) auth=(A_username, A_password))
if response.status_code == 200:
# Check the Content-Length header to get the expected file size
expected_size = int(response.headers.get("Content-Length", 0))
# Write the downloaded content to a file
with open('mate0030.17o.Z', 'wb') as f:
for chunk in response.iter_content(chunk_size=1000):
if chunk:
f.write(chunk)
print(f"Downloaded {len(chunk)} bytes...")
# Check the actual size of the downloaded file
actual_size = os.path.getsize('mate0030.17o.Z')
# Compare the expected and actual sizes to ensure the file was fully downloaded
if actual_size == expected_size:
print("File downloaded successfully!")
else:
print("File download incomplete.")
else:
print("Unable to download file.")
output of the code
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 1000 bytes...
Downloaded 178 bytes...
File download incomplete.
I have queries as listed below
1)I am able to access the file with my python code, but my data down is going incomplete. why?
2)How should the content of .netrc file should be feeded to my code? Here I am feeding the user credentials manually as I have no idea of interfacing this details.
Please suggest.