download the modis data via Appeears

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
kson
Posts: 1
Joined: Mon Jul 25, 2022 4:24 pm America/New_York
Answers: 0

download the modis data via Appeears

by kson » Mon Jul 25, 2022 4:27 pm America/New_York

I have tried to download modis data with R script below. I had no problem downloading the files but now I can get the following error messages.
I would appreciate it if anyone helps me out.

dataPath <- "C:/Project/Columbia_SWAT/NHD_YRB_SWAT/codes/MODIS_Fire/data/et"

downloadList <-"C:/Project/Columbia_SWAT/NHD_YRB_SWAT/codes/MODIS_Fire/data/et/YRBETPET20002022-download-list.txt"

filesl <- scan(downloadList, what='list', sep='\n')
Read 5060 items
> for (d in 1:length(filesl)){
+
+
+ curl_download(url=filesl[d], destfile=paste0(dataPath,"/",basename(filesl[d])),quiet =F, handle =new_handle())
+ Sys.sleep(1)
+ # print(paste0('Downloading source data (Step 1 of 6): ',round(d /length(filesl) * 100, digits=2),'%'))
+ print(d/length(filesl))
+ }
[100%] Downloaded 138 bytes...
Error in curl_download(url = filesl[d], destfile = paste0(dataPath, "/", :
HTTP error 403.
>

Tags:

LP DAAC - mahsajami
Subject Matter Expert
Subject Matter Expert
Posts: 3
Joined: Thu Apr 02, 2020 1:15 pm America/New_York
Answers: 0

Re: download the modis data via Appeears

by LP DAAC - mahsajami » Wed Jul 27, 2022 9:13 am America/New_York

Hello,
Thanks for reaching out. A NASA Earthdata Login account is required to login to the AppEEARS API and download outputs of a request. Your username and password should be used to create an authentication token. Adding this temporary token to handle argument in curl_download function, enables you to make a successful call to AppEEARS API and download your outputs. please see the updated version of your script below for more details.
library(httr)
library(jsonlite)
library(curl)
# Create a token by calling AppEEARS API login service. Update the “USERNAME” and “PASSEWORD” with yours below
secret <- base64_enc(paste("USERNAME", "PASSWORD", sep = ":"))
response <- POST("https://appeears.earthdatacloud.nasa.gov",
add_headers("Authorization" = paste("Basic", gsub("\n", "", secret)),
"Content-Type" = "application/x-www-form-urlencoded;charset=UTF-8"),
body = "grant_type=client_credentials")
token_response <- prettify(toJSON(content(response), auto_unbox = TRUE))

# Create a handle
s = new_handle()
handle_setheaders(s, 'Authorization'=paste("Bearer", fromJSON(token_response)$token))

# Loop through the URLs and downlaod outputs
for (d in 1:length(filesl)){
curl_download(url=filesl[d], destfile=paste0(dataPath,"/",basename(filesl[d])), handle = s)
Sys.sleep(1)
print(paste0("Downloading ", d, " out of ", length(filesl)))
}
Please let us know if you still have an issue downloading your outputs.

saisrikumar
Posts: 1
Joined: Fri Jul 29, 2022 3:27 am America/New_York
Answers: 0

Re: download the modis data via Appeears

by saisrikumar » Fri Jul 29, 2022 3:34 am America/New_York

hi sir i trying to download the data but its not working from 27.07.2022 from this link (https://e4ftl01.cr.usgs.gov/MOLA/MYD13Q1.006/) & R
> MODIStsp()
GDAL version in use: 3.0.4
[Fri Jul 29 12:29:38 2022] MODIStsp --> Starting processing
[Fri Jul 29 12:29:38 2022] Accessing http server at: https://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.006/
Error in curl::curl_fetch_memory(url, handle = handle): Timeout was reached: [e4ftl01.cr.usgs.gov] Operation timed out after 10000 milliseconds with 0 out of 0 bytes received
Request failed [ERROR]. Retrying in 1 seconds...
Set Reprocess to "Yes" to reprocess existing data!
[Fri Jul 29 12:30:19 2022] Accessing http server at: https://e4ftl01.cr.usgs.gov/MOLA/MYD13Q1.006/
[Fri Jul 29 12:30:22 2022] Retrieving list of available ` Aqua ` Files for Year 2022
Error in curl::curl_fetch_memory(url, handle = handle): Timeout was reached: [e4ftl01.cr.usgs.gov] Operation timed out after 10013 milliseconds with 0 out of 0 bytes received
Request failed [ERROR]. Retrying in 5.2 seconds...
Request failed [ERROR]. Retrying in 9 seconds...
Timeout was reached: [e4ftl01.cr.usgs.gov] Operation timed out after 10004 milliseconds with 0 out of 0 bytes received
>

LP DAAC - jwilson
User Services
User Services
Posts: 268
Joined: Mon Sep 30, 2019 12:39 pm America/New_York
Answers: 1
Has thanked: 9 times

Re: download the modis data via Appeears

by LP DAAC - jwilson » Mon Aug 01, 2022 10:57 am America/New_York

We are experiencing a higher than average load on the data pool after our planned maintenance which is causing time out issues. It is up and running, so please continue to try accessing it throughout the day.
If you continue to have issues, please contact lpdaac@usgs.gov

LP DAAC - dgolon
User Services
User Services
Posts: 287
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 16 times
Been thanked: 2 times
Contact:

Re: download the modis data via Appeears

by LP DAAC - dgolon » Tue Aug 02, 2022 1:17 pm America/New_York

Hi @saisrikumar It looks like we are seeing more movement in the Data Pool now. Please clear your cache and try your request again.
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

Post Reply