wget error 403

Use this Forum to find information on, or ask a question about, NASA Earth Science data.
Post Reply
oceani
Posts: 79
Joined: Wed Aug 23, 2006 12:32 pm America/New_York
Answers: 0
Has thanked: 2 times

wget error 403

by oceani » Tue Dec 03, 2019 2:30 pm America/New_York

I am in the process of modifying our scripts that download our subscription data from your server using the new file naming convention.  All is working well, except for one subscription number (1385).  The command I am using to download the list of available files for subscription 1385 to a file (muamf1) is:wget --post-data="subID=1385&results_as_file=1" -O - https://oceandata.sci.gsfc.nasa.gov/api/file_search > /users/staff/nasamodissst/munetcdf/muamf1For this ID only I am getting the following error:--2019-12-03 12:16:25--  https://oceandata.sci.gsfc.nasa.gov (/search/file_search.cgi)   Resolving oceandata.sci.gsfc.nasa.gov... xx.xxx.xx.xx   Connecting to oceandata.sci.gsfc.nasa.gov|xx.xxx.xx.xx|:443... connected.   HTTP request sent, awaiting response... 403 Forbidden   2019-12-03 12:16:25 ERROR 403: Forbidden.My other subscription IDs work fine using the same wget command string.  Please let me know what I may be doing wrong in this case.Thanks,Mark

Tags:

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

wget error 403

by OB.DAAC - SeanBailey » Tue Dec 03, 2019 3:16 pm America/New_York

Well, it worked for me, so I see nothing wrong with the call.

Silly question...have you tried again?

Sean

oceani
Posts: 79
Joined: Wed Aug 23, 2006 12:32 pm America/New_York
Answers: 0
Has thanked: 2 times

wget error 403

by oceani » Tue Dec 03, 2019 4:05 pm America/New_York

Yep.  Tried about 6-7 times.  Just tried again a few minutes ago.  That is quite strange.  The exact same call works (so far ... still updating) for all of my other subscriptions. Could that folder containing those subscription files have different permissions than the rest?  Perhaps, I should remove that subscription and create a new one???

Thanks for the look.  Let me know if you have any other suggestions.

Mark

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

wget error 403

by OB.DAAC - SeanBailey » Tue Dec 03, 2019 5:54 pm America/New_York

Mark,

I, too,  just tried again from another machine on a completely different network...still works.

There is no folder for the non-extracted subscription.  In fact, if there are 30 files in your retrieval, they could reside on 30 different machines.
No need to recreate the subscription, it would have no effect on the API - other than giving you a different subscription ID to pull.

All I can suggest is that perhaps the command you put in the forum post is not be the command that's being executed.

Sean

oceani
Posts: 79
Joined: Wed Aug 23, 2006 12:32 pm America/New_York
Answers: 0
Has thanked: 2 times

wget error 403

by oceani » Tue Dec 03, 2019 6:35 pm America/New_York

OK.  I actually copied the command from my script file and pasted to the post.  I also copied the wget command from a script that is working and just swapped out the subscription ID number.  Same result.  I'll see what I can figure out. I'll try from a different machine as well.

Thanks,

Mark

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

wget error 403

by OB.DAAC - SeanBailey » Tue Dec 03, 2019 6:51 pm America/New_York

Very, very odd indeed...
Mind providing the IP you're coming from? Maybe there's a hint in the log files here...

Sean

oceani
Posts: 79
Joined: Wed Aug 23, 2006 12:32 pm America/New_York
Answers: 0
Has thanked: 2 times

wget error 403

by oceani » Wed Dec 04, 2019 5:53 pm America/New_York

xx.xxx.xx.xx

OB ODPS - jgwilding
Subject Matter Expert
Subject Matter Expert
Posts: 139
Joined: Fri Feb 19, 2021 1:09 pm America/New_York
Answers: 0
Been thanked: 1 time

wget error 403

by OB ODPS - jgwilding » Wed Dec 04, 2019 9:20 pm America/New_York

Mark,

What version of wget do you have, and do you have different results with the GET method?  Note quotes around URL become necessary to protect the special chars.

?wget -O - "https://oceandata.sci.gsfc.nasa.gov/api/file_search?subID=1385&results_as_file=1"

My version of wget is, GNU Wget 1.17.1 built on linux-gnu.

Was also wondering if this subscription is Nth in a script of successive calls, where N is somehow exceeding a frequency limit.  Probably not it as i assume you've tried just the one subscription URL outside of the script.

john

OB.DAAC - SeanBailey
User Services
User Services
Posts: 1464
Joined: Wed Sep 18, 2019 6:15 pm America/New_York
Answers: 1
Been thanked: 5 times

wget error 403

by OB.DAAC - SeanBailey » Fri Dec 06, 2019 6:10 pm America/New_York

I can see the 403s in the logs for your IP (well, IPs in the same last octet), but nothing to indicate why.
Are you still seeing issues?
Have you tried John's suggestion of using the "GET" method instead of "POST"?

Sean

Post Reply