×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

403 Forbidden/LWP UserAgent - please help

403 Forbidden/LWP UserAgent - please help

403 Forbidden/LWP UserAgent - please help

(OP)
Hello Tek-Tippers,

I am having a problem accessing an asp.net site and get a 403 Forbidden error with my script. This does not happen to any html type website so I am asking you for help to scrape asp.net site.

Thank you in advance for all the help.


use LWP::Debug qw(+);
use LWP::UserAgent;
use HTTP::Cookies;


my $ua = new LWP::UserAgent;
my $url = 'http://www.site.com/index.aspx';
my $cookie_jar = HTTP::Cookies->new;


$ua->proxy("http", "proxysite:80");

my $req = new HTTP::Request GET => $url;
my $res = $ua->request($req);
$req = $res->request();
$cookie_jar->extract_cookies($res);

print "\nREQUEST-HEADERS\n";
print $req->headers_as_string();
print "\nREQUEST-CONTENT\n";
print $req->content;
if ($res->is_success) {
print "\nRESPONSE-HEADERS\n";
print $res->headers_as_string();
print "\nRESPONSE-CONTENT\n";
print $res->content;
} else {
print "\nRESPONSE-ERROR\n";
print $res->error_as_HTML();
}


Output:
REQUEST-HEADERS
User-Agent: libwww-perl/5.833

REQUEST-CONTENT

RESPONSE-ERROR
<html>
<head><title>An Error Occurred</title></head>
<body>
<h1>An Error Occurred</h1>
<p>403 Forbidden</p>
</body>
</html>

RE: 403 Forbidden/LWP UserAgent - please help

(OP)
That worked, thank you!

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close