obdaac_download.py re-downloads data already existing locally without "--force"

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
ywystu
Posts: 6
Joined: Sat Oct 29, 2022 12:25 pm America/New_York
Answers: 0

obdaac_download.py re-downloads data already existing locally without "--force"

by ywystu » Tue Dec 05, 2023 9:44 pm America/New_York

I downloaded data from OB.DAAC with obdaac_download.py and ran the script with the command: python3 ${dir}/obdaac_download.py -v --http_manifest ${i} --odir ./${i:87:16}/ --uncompress
Note that I ran without "--force". However, the data already existing locally was been re-downloaded.
Something is strange: it worked right some months ago, and now it doesn't work normally.
I also tried to run the script on Win10 OS on other machine with (python %dir%\obdaac_download.py -v --http_manifest "!url!" --odir .\!url:~87,16!\ --uncompress).
What's wrong with it?!

Attached below is my entire code:
--------------------------------------------------------------------------
http_manifest=$(cat download.list)
dir=$(pwd)

for i in ${http_manifest[*]};
do
cd $dir/down
{
python3 ${dir}/obdaac_download.py -v --http_manifest ${i} --odir ./${i:87:16}/ --uncompress
rm -f ./http_manifest.txt
}||{
echo "${i:87:16} met an error!"
continue
}

done

echo
read -p " " a

Tags:

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1461
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 4 times

Re: obdaac_download.py re-downloads data already existing locally without "--force"

by OB.DAAC - SeanBailey » Wed Dec 06, 2023 1:46 pm America/New_York

There was an issue where the last-modified header wasn't being sent, so the code could not do the comparison with the file on disk.

This has been fixed.

Thanks for alerting us to the issue,

Sean

Post Reply