How to Configure and Confirm Download Methods in Earthdata Search

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
Earthdata - wxedward
User Services
User Services
Posts: 156
Joined: Wed Jan 17, 2024 10:24 am America/New_York
Answers: 22

How to Configure and Confirm Download Methods in Earthdata Search

by Earthdata - wxedward » Wed Jan 31, 2024 3:50 pm America/New_York

Earthdata Search enables users to download data through various methods. These methods include:

Direct Download - Direct download of all data associated with the selected granules. The desired data will be available for download immediately. Files will be accessed from a list of links displayed in the browser or by using a download script. Stage for Delivery - Submit a request for data to be staged for delivery. Data files will be compressed in zip format and stored for retrieval via HTTP. The user will receive an email from the data provider when the files are ready to download. Customized - Select options like variables, transformations, and output formats to customize the data. The desired data files will be made available for access after the data provider has finished processing the request. The user will receive an email from the data provider when the files are ready to download.


STEP-BY-STEP INSTRUCTIONS

Configure Download Methods for Collections

Configure Download Methods for Collections
Image These steps can be used to enable Stage for Delivery or Customized download options.
  1. LOGIN to the Metadata Management Tool at https://mmt.earthdata.nasa.gov.
    Image This method will enable download options in the PROD environment only. For changes to UAT use https://mmt.uat.earthdata.nasa.gov/. For changes to SIT use https://mmt.sit.earthdata.nasa.gov/.
    Image
  2. SELECT the appropriate provider in both the context drop-down and under "Provider." SELECT the Services radio button, then CLICK Search Services.
    Image
  3. CLICK the appropriate service implementation record.
    Image The service implementation options will vary by data provider.
    Image
  4. CLICK on Manage Collection Associations.
    Image
  5. To add a collection for this service implementation record, CLICK on Add Collection Associations.
    Alternatively, to delete a collection for this service implementation record, SELECT the appropriate collection by marking the checkbox next to the entry title, then CLICK Delete Selected Associations.
    Image Check for existing collection associations by SEARCHING the records for the collection entry title or short name. This is most easily done by using the Command+F function on Mac, or Control+F function on Windows. Please note, there may be several pages of collections sorted in alphabetical order.
    Image
  6. SELECT a Search Field and ENTER the appropriate Search Term to find the collection of interest. CLICK the + button to add each collection of interest. Once all collections have been added, CLICK Submit
    Image The Earthdata Search background job that searches for new or updated collection associations runs every 4 hours, and takes approximately 45 minutes to complete. It may take some time before you see any changes on the Earthdata Search website.
    Image
Confirm Existing Download Methods for Collections


Confirm Existing Download Methods for Collections
Image These steps can be done in a web browser, using the terminal or command line, or by using a dedicated application for API environments (i.e. Postman). These instructions will be shown using a web browser.

  1. Using the following URL format, NAVIGATE to the page below in a web browser. https://cmr.earthdata.nasa.gov/search/collections.json?ECHO_collection_id=<ECHO_collection_id>&include_tags=* Example for concept ID C1299783579-LPDAAC_ECS: https://cmr.earthdata.nasa.gov/search/collections.json?ECHO_collection_id=C1299783579-LPDAAC_ECS&include_tags=*
    Image This example uses the echo_collection_id parameter to find the appropriate collection. The list of available parameters to search for in the CMR API can be found at https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html.
    Image
  2. RETRIEVE a token (if necessary). Some collections and/or data providers require a token to access the CMR API information. In this case, the response returned from CMR would be blank. An example would be concept ID C1236303843-NSIDC_ECS.
    Image

    In this case the URL format should be as follows, to include a token:
    https://cmr.earthdata.nasa.gov/search/collections.json?ECHO_collection_id=<ECHO_collection_id>&include_tags=*&token=<token>
    A token can be retrieved using the Postman POST Request file below. In the Body of the request, replace the username and password with your Earthdata Login credentials.
  3. Various tags for the collection can be found in the metadata, depending on the download method of interest.

    Direct Download

    Direct Download
    An example collection for Direct Download would be concept ID C194001241-LPDAAC_ECS. The tag associations for direct download can be found under:
    edsc.extra.serverless.collection_capabilities
    granule_online_access_flag
    https://cmr.earthdata.nasa.gov/search/collections.json?ECHO_collection_id=C194001241-LPDAAC_ECS&include_tags=*
    Image


    The granule_online_access_flag is set by granule metadata within the collection. The granule(s) must have an OnlineAccessURL tag in its metadata. Earthdata Search inspects the first granule for this OnlineAccessURL tag that is then used to enable Direct Download for the collection.
    An example for this same collection would be granule concept ID G1658148046-LPDAAC_ECS.
    https://cmr.earthdata.nasa.gov:443/search/concepts/G1658148046-LPDAAC_ECS/1
    Image

    Stage for Delivery

    Stage for Delivery
    An example collection for Stage for Delivery would be concept ID C1299783579-LPDAAC_ECS. The tag associations for the ECHO orders service implementation can be found under edsc.extra.serverless.subset_service.ECHO_orders.
    https://cmr.earthdata.nasa.gov/search/collections.json?ECHO_collection_id=C1299783579-LPDAAC_ECS&include_tags=*
    The id field under the data tag displays the service implementation ID. In this example, the service implementation ID is S1568897323-LPDAAC_ECS.
    Using the steps above under 'Configure Download Methods for Collections,' this ID number will correlate with the MMT service implementation record ID. This ID can be seen in the URL of the MMT Service Implementation page.
    https://mmt.earthdata.nasa.gov/services/S1568897323-LPDAAC_ECS
    Image

    Customized

    Customized
    An example collection for Customized Download would be C43677706-LARC. The tag associations for the ESI service implementation can be found under edsc.extra.serverless.subset_service.esi. https://cmr.earthdata.nasa.gov/search/collections.json?ECHO_collection_id=C43677706-LARC&include_tags=*
    Image


    The following tags determine the various badges seen on the Earthdata Search collections page. The tags are generated by CMR, referencing the UMM-S record:
    • has_spatial_subsetting
    • has_transforms
    • has_variables
    • has_formats
    • has_temporal_subsetting
    Image


    The id field under service_option_definitions is the ID for the ECHO form used to generate the customized options available on the Earthdata Search project page.
    In this example the ECHO form ID is 15E07651-0061-8ACF-75BF-81B91D9FFFB0. The name of the ECHO form is MIL2ASAE.2 ESI Service.
    Image
  4. CONFIRM the appropriate Earthdata Search tags are associated with the collection. If the appropriate tags for the associated download method are missing, the download option on Earthdata Search will not be made available.
    Image The Earthdata Search background job that searches for new or updated collection associations runs every 4 hours, and takes approximately 45 minutes to complete. It may take some time before you see any changes on the Earthdata Search website.
    • Check the UMM-S Record for Collections
    Data providers can enable any and all of these download options in Earthdata Search by following the step-by-step instructions below. Stage for Delivery and Customized Downloads require more manual configuration. At a high level, in order to enable data access for one of these methods, Earthdata Search requires a collection be associated with both a UMM-S record and an echoform. Please make sure you carefully follow the steps below to ensure your collections are configured correctly. This guide will cover the following information:

    Configure Download Methods for Collections Confirm Existing Download Methods for Collections

Tags:

Post Reply