Bulk Download Problem

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
amirhossein
Posts: 3
Joined: Tue Jul 04, 2023 9:13 am America/New_York
Answers: 0

Bulk Download Problem

by amirhossein » Tue Jul 04, 2023 9:16 am America/New_York

Hello everyone,
When I copy the MODIS-AOD text file links into the Free Download Manager to download links automatically (Links come from Search Earth Data) returns error. Does anyone have any solution?

Tags:

LAADS_UserServices_M
User Services
User Services
Posts: 274
Joined: Mon Sep 30, 2019 8:33 am America/New_York
Answers: 1
Has thanked: 1 time

Re: Bulk Download Problem

by LAADS_UserServices_M » Wed Jul 05, 2023 11:11 am America/New_York

From where are you trying to download data, ladsweb, nrt, or Earthdata Cloud? Can you provide some example URLs to investigate?
Regards,
LAADS User Services

To receive news from LAADS DAAC direct to your inbox, email laadsdaac-join@lists.nasa.gov with “subscribe” in the subject line.

amirhossein
Posts: 3
Joined: Tue Jul 04, 2023 9:13 am America/New_York
Answers: 0

Re: Bulk Download Problem

by amirhossein » Thu Jul 06, 2023 2:19 am America/New_York

Thanks for getting back to me. Here is a sample link from MODIS vegetation product.
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/MOD13Q1.061/MOD13Q1.A2020353.h23v05.061.2021012024119/MOD13Q1.A2020353.h23v05.061.2021012024119.hdf
Although my Earth Data account has already authorized for bulk download, still I have same problem for all products coming from https://search.earthdata.nasa.gov/search
However the products from https://giovanni.gsfc.nasa.gov/giovanni/ or https://appeears.earthdatacloud.nasa.gov/ downloaded automatically without any issues.

LP DAAC - dgolon
User Services
User Services
Posts: 274
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 15 times
Been thanked: 2 times
Contact:

Re: Bulk Download Problem

by LP DAAC - dgolon » Thu Jul 06, 2023 9:12 am America/New_York

Hi @amirhossein A few users have reported issues accessing the MOD13Q1 data outside of the LP's Data Pool. We are looking into this issue and we will report back when we have more information. Thanks - Danielle
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

LP DAAC - dgolon
User Services
User Services
Posts: 274
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 15 times
Been thanked: 2 times
Contact:

Re: Bulk Download Problem

by LP DAAC - dgolon » Thu Jul 13, 2023 9:57 am America/New_York

Hi @amirhossein Could you please let us know which download manager tool are you using? If interested, we have a few resources we could direct you to, to download data directly from the command line. Thanks, Danielle
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

amirhossein
Posts: 3
Joined: Tue Jul 04, 2023 9:13 am America/New_York
Answers: 0

Re: Bulk Download Problem

by amirhossein » Fri Jul 14, 2023 2:54 am America/New_York

Here you are:
https://www.freedownloadmanager.org/

The download manager works well in NASA Giovanni but only returns error in Search EarthData

LP DAAC - dgolon
User Services
User Services
Posts: 274
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 15 times
Been thanked: 2 times
Contact:

Re: Bulk Download Problem

by LP DAAC - dgolon » Tue Jul 25, 2023 10:25 am America/New_York

Hi @amirhossein We have not used that tool before and did not create it, so unfortunately we can't help answer you question and recommend you reach out to the contact of the team that created that tool.

Alternatively, we recommend using bulk downloading using wget:

To bulk download your outputs using wget:

First, save the download list for files you want to download. If you want all the outputs, select all and then click on “Save Download List”.

Next, type the line below in command line and press enter to get a token:
(Insert your Earthdata Login username and password in the red parts.)
> wget -q -O - --method POST --user=Insert_Your_EDL_Username --password= Insert_Your_EDL_Password--auth-no-challenge https://appeears.earthdatacloud.nasa.gov/api/login

Your token should look like:
{"token_type": "Bearer", "token": "r0HkNQtYquKjkOZbY-6P8mgjA8....", "expiration": "2023-05-04T14:05:47Z"}

To download the files, type the line below in your command line and press enter:
Copy your token (purple above) and paste it instead of Insert_Your_Token below.

Also insert the full path to the saved downloaded list below (Insert_full_Path_to_Your_Download_List)

> wget --header "Authorization: Bearer Insert_Your_Token” -i Insert_full_Path_to_Your_Download_List
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

Post Reply