Page 1 of 1

obdaac_download.py re-downloads data already existing locally without "--force"

Posted: Tue Dec 05, 2023 9:44 pm America/New_York
by ywystu
I downloaded data from OB.DAAC with obdaac_download.py and ran the script with the command: python3 ${dir}/obdaac_download.py -v --http_manifest ${i} --odir ./${i:87:16}/ --uncompress
Note that I ran without "--force". However, the data already existing locally was been re-downloaded.
Something is strange: it worked right some months ago, and now it doesn't work normally.
I also tried to run the script on Win10 OS on other machine with (python %dir%\obdaac_download.py -v --http_manifest "!url!" --odir .\!url:~87,16!\ --uncompress).
What's wrong with it?!

Attached below is my entire code:
--------------------------------------------------------------------------
http_manifest=$(cat download.list)
dir=$(pwd)

for i in ${http_manifest[*]};
do
cd $dir/down
{
python3 ${dir}/obdaac_download.py -v --http_manifest ${i} --odir ./${i:87:16}/ --uncompress
rm -f ./http_manifest.txt
}||{
echo "${i:87:16} met an error!"
continue
}

done

echo
read -p " " a

Re: obdaac_download.py re-downloads data already existing locally without "--force"

Posted: Wed Dec 06, 2023 1:46 pm America/New_York
by OB.DAAC - SeanBailey
There was an issue where the last-modified header wasn't being sent, so the code could not do the comparison with the file on disk.

This has been fixed.

Thanks for alerting us to the issue,

Sean