Downloading MOD13A1 and MYD13A1 files - methods?

[binxu748er]

Hello everyone, I'm very new to this site and need it for my research.

I seem to have the problem, when downloading all the files at once using a python code from the saved file.txt with the links to the files.hdf, all downloaded files turn out to be "corrupted"/unusable. However, if I download each file manually, the files seems to be fine.

Same problem with the cygwin method.

Is there a quicker way to download the files? I have to download roughly 2000 files

Many thanks!

[LP DAAC - lien]

Hello,
Could you attach your python code and we can take a look, Also, could you attach the text file then we can see where you compile it from. If you would like you could send it to the LPDAAC email: lpdaac@usgs.gov
Thanks,
Brett

[binxu748er]

Hello,
Py code:

import requests
from tqdm import tqdm

def download_hdf_files(links_file, download_path):
with open(links_file, 'r') as file:
links = file.readlines()

for link in tqdm(links, desc="Downloading files...", unit="file"):
link = link.strip()
filename = link.split("/")[-1]
filepath = f"{download_path}/{filename}"


response = requests.get(link, stream=True)
total_size = int(response.headers.get('content-length', 0))

with open(filepath, 'wb') as file, tqdm(
desc=filename,
total=total_size,
unit='B',
unit_scale=True,
unit_divisor=1024,
) as bar:
for data in response.iter_content(chunk_size=1024):
bar.update(len(data))
file.write(data)

if __name__ == "__main__":
#Textfile
links_file_path = "C:/folder/file.txt..."

#target folder
download_directory = "C:/folder2/..."

download_hdf_files(links_file_path, download_directory)


Text file (converted to PDF for upload purpose) is attached. I have only uploaded one txt (PDF) file for MOD13A1 (549 entries). The procedure for MYD13A1 and other files with albedo data is the same.

[LP DAAC - dgolon]

Hello @binxu748er I spoke with our science team and they recommended using the wget to bulkdown the data. You can find more information here: https://github.com/nasa/LPDAAC-Data-Resources/blob/main/guides/bulk_download_using_wget.md Thank you -- Danielle

[binxu748er]

Hello, it worked now. Many thanks for the support.