Cannot resolve domain using wget.

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
zbegg
Posts: 4
Joined: Sun Jan 13, 2019 9:58 pm America/New_York
Answers: 0

Cannot resolve domain using wget.

by zbegg » Wed Nov 29, 2023 4:50 pm America/New_York

Hi,
I am using wget to download datasets using api key from AWS instance, we notice that we are not able to resolve the domain name, whilst this works well from my work machine.

Any leads on this will be appreciated.

Thanks,

Filters:

LP DAAC - dgolon
User Services
User Services
Posts: 311
Joined: Mon Sep 30, 2019 10:00 am America/New_York
Answers: 0
Has thanked: 23 times
Been thanked: 2 times
Contact:

Re: Cannot resolve domain using wget.

by LP DAAC - dgolon » Thu Nov 30, 2023 9:17 am America/New_York

Hi @zbegg To help us look into the issue more, could you please tell us which datasets are you trying to download? Thanks -- Danielle
Subscribe to the LP DAAC listserv by sending a blank email to lpdaac-join@lists.nasa.gov.

Sign up for the Landsat listserv to receive the most up to date information about Landsat data: https://public.govdelivery.com/accounts/USDOIGS/subscriber/new#tab1.

zbegg
Posts: 4
Joined: Sun Jan 13, 2019 9:58 pm America/New_York
Answers: 0

Re: Cannot resolve domain using wget.

by zbegg » Tue Dec 05, 2023 6:56 pm America/New_York

Hi, I am using this command to download the data,

wget -nc https://oceandata.sci.gsfc.nasa.gov/ob/getfile/AQUA_MODIS.20230821.L3m.DAY.CHL.chlor_a.4km.nc?appkey={my_api_key} --no-check-certificate -O AQUA_MODIS.20230821.L3m.DAY.CHL.chlor_a.4km.nc

Error:
Resolving oceandata.sci.gsfc.nasa.gov...
169.154.128.84, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov|169.154.128.84|:443... failed: Connection timed out.
Connecting to oceandata.sci.gsfc.nasa.gov|2001:4d0:2418:128::84|:443... failed: Network is unreachable.

This server is sitting in amazon AWS cloud.

Thanking you in advance.

OB General Science - guoqingw
Subject Matter Expert
Subject Matter Expert
Posts: 79
Joined: Fri Jun 03, 2022 10:54 am America/New_York
Answers: 0
Location: NASA GSFC
Been thanked: 1 time
Contact:

Re: Cannot resolve domain using wget.

by OB General Science - guoqingw » Thu Dec 07, 2023 10:05 am America/New_York

Please email "connection_problems@oceancolor.gsfc.nasa.gov" with the public ip address you are connection from, the time you tried to connect and a short description of the network issue you are experiencing. The NASA OBPG network team will assist you.

OB.DAAC - amscott
User Services
User Services
Posts: 356
Joined: Mon Jun 22, 2020 5:24 pm America/New_York
Answers: 1
Has thanked: 8 times
Been thanked: 4 times

Re: Cannot resolve domain using wget.

by OB.DAAC - amscott » Mon Dec 11, 2023 5:34 pm America/New_York

Although I see the file via file search, it does appear that the file is missing, resulting in an error page. I've alerted the data production team of this issue.

westernoverseasaustralia
Posts: 1
Joined: Tue Dec 12, 2023 12:32 am America/New_York
Answers: 0

Re: Cannot resolve domain using wget.

by westernoverseasaustralia » Tue Dec 12, 2023 12:45 am America/New_York

If you are experiencing issues resolving a domain using wget, there are a few things you can check and troubleshoot. Here are some steps you can take:

Check Internet Connection:
Ensure that your internet connection is stable and working. You can try opening a website in a web browser to verify connectivity.

DNS Resolution:
Confirm that the domain name can be resolved to an IP address. You can use the nslookup or dig command to check DNS resolution. For example:


nslookup website domain name
DNS Server Configuration:
Verify that your DNS server settings are correct. You can check the DNS configuration in your system's network settings or in the configuration files, depending on your operating system.

Firewall or Proxy Settings:
Make sure that there are no firewall or proxy settings blocking the wget command. If you are behind a proxy, you may need to configure wget to use the proxy.

Check wget Command:
Ensure that the wget command is correctly formatted. Here is a basic example:


wget website domain name
HTTP vs HTTPS:
Verify if the website uses HTTPS. If it does, make sure to use https:// in the URL when using wget. Some websites may automatically redirect HTTP to HTTPS.


wget website domain name
User-Agent:
Some websites may block requests without a proper User-Agent header. You can try setting a User-Agent using the -U or --user-agent option:


wget -U "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3" http://example.com
SSL/TLS Issues:
If the website uses HTTPS, there might be SSL/TLS-related issues. Make sure your system has the necessary certificates installed.

Network Configuration:
Check if there are any network configuration issues on your system, such as incorrect DNS settings or routing problems.

Debugging Output:
Use the --debug option with wget to get more detailed information about what might be causing the issue:


wget --debug website domain name
By going through these steps, you should be able to identify and resolve the issue preventing wget from resolving the domain. If the problem persists, there might be specific details about the error that can help in further troubleshooting.
Last edited by westernoverseasaustralia on Tue Dec 12, 2023 12:55 am America/New_York, edited 1 time in total.

OB.DAAC - amscott
User Services
User Services
Posts: 356
Joined: Mon Jun 22, 2020 5:24 pm America/New_York
Answers: 1
Has thanked: 8 times
Been thanked: 4 times

Re: Cannot resolve domain using wget.

by OB.DAAC - amscott » Wed Dec 20, 2023 4:26 pm America/New_York

Hello,

After re-reading this, I think the issue may be your appkey. Please try your wget command with an appkey created here: https://oceandata.sci.gsfc.nasa.gov/appkey/

Our team has tested and at least 3 of us have been able to pull the desired file down by repeating your wget command from our machines.

Post Reply