Connect to a forum to get new threads

[titosat]

Hi i am new to perl I start with perl 3 days ago. I am trying to do a script that connect to a forum and get the latest thread do the same every five minutes and verify if the new thread is not the same as the last one it got from the last login and then save it on a text file (i realy want to send it to mysql database on other comp but i will work on that later)on the new threads info i want to get (description, by who, views, reply, and the link to the thread). I have some thin working but i don't know how to login.

 # Create a user agent object
  use LWP::UserAgent;
  $ua = LWP::UserAgent->new;
  $ua->agent("MyApp/0.1 ");

  # Create a request
  my $req = HTTP::Request->new(POST => '[URL unfurl="true"]http://www.totalfta.com');[/URL]
  $req->content_type('application/x-[URL unfurl="true"]www-form-urlencoded');[/URL]
  $req->content('query=lib[URL unfurl="true"]www-perl&mode=dist');[/URL]

  # Pass request to the user agent and get a response back
  my $res = $ua->request($req);

  # Check the outcome of the response
  if ($res->is_success) {
  my($text) = $res->content;
  if ( $text =~ /<a href="(.+)" title="Go to first unread     post in thread '(.+)'"><strong>(.+)<\/strong><\/a>/m ) {
     print "[URL unfurl="true"]http://www.totalfta.com/$1,[/URL] $2, $3\n";
     
  }
 }
 else {
     print $res->status_line, "\n";
  }

 

[titosat]

Hi again I find this script on the web and is working but when It try to log i get the msg that the username or password are wrong and i try to log using the internet explorer to verify that the user and pass where working and it loging with no problen.

use strict;
use warnings;
use HTTP::Cookies;
use LWP::UserAgent;
use HTTP::Request::Common qw(POST);

my $ua = new LWP::UserAgent;
my $cookie_jar = HTTP::Cookies->new(file => 'lwpcookies.txt',
  autosave => 1, ignore_discard => 1);
$ua->cookie_jar($cookie_jar);

my $url = '[URL unfurl="true"]http://www.totalfta.com/login.php';[/URL]

my $req = POST $url, Content_Type => 'form-data', Content => [ 'do' => 'login', UserName => 'some_user', Password => 'some_pass' ];
my $res = $ua->request($req);

my $content = $res->as_string;
$content =~ s/<tr[^>]*>/\n/gs;  # scrub the tags etc.
$content =~ s/<[^>]*>//gs;
$content =~ s/[ \t][ \t]+/ /gs;
$content =~ s/\n\s+/\n/gs;
$content =~ s/\s+\n/\n/gs;
$content =~ s/\n\n+/\n/gs;
print $content, "\n";

__END__

 

[titosat]

Hi again I find this script on the web and is working but when It try to log i get the msg that the user name or password are wrong and i try to log using the internet explorer to verify that the user and pass where working and it login with no problem. And i what to add the do=getnew to it to get the new threads "

http://www.totalfta.com/search.php?do=getnew".

thz in advance

use strict;
use warnings;
use HTTP::Cookies;
use LWP::UserAgent;
use HTTP::Request::Common qw(POST);

my $ua = new LWP::UserAgent;
my $cookie_jar = HTTP::Cookies->new(file => 'lwpcookies.txt',
  autosave => 1, ignore_discard => 1);
$ua->cookie_jar($cookie_jar);

my $url = '[URL unfurl="true"]http://www.totalfta.com/login.php';[/URL]

my $req = POST $url, Content_Type => 'form-data', Content => [ 'do' => 'login', UserName => 'some_user', Password => 'some_pass' ];
my $res = $ua->request($req);

my $content = $res->as_string;
$content =~ s/<tr[^>]*>/\n/gs;  # scrub the tags etc.
$content =~ s/<[^>]*>//gs;
$content =~ s/[ \t][ \t]+/ /gs;
$content =~ s/\n\s+/\n/gs;
$content =~ s/\s+\n/\n/gs;
$content =~ s/\n\n+/\n/gs;
print $content, "\n";

__END__

 

[KevinADC]

are you using your real name and password in the above code?

- Kevin, perl coder unexceptional!

 

[titosat]

yes

 

[KevinADC]

well, the form for the login at the site is this:

		<!-- login form -->
		<form action="login.php?do=login" method="post" onsubmit="md5hash(vb_login_password, vb_login_md5password, vb_login_md5password_utf, 0)">
		<script type="text/javascript" src="clientscript/vbulletin_md5.js?v=364"></script>

		<table cellpadding="0" cellspacing="3" border="0">
		<tr>
			<td class="smallfont"><label for="navbar_username">User Name</label></td>
			<td><input type="text" class="bginput" style="font-size: 11px" name="vb_login_username" id="navbar_username" size="10" accesskey="u" tabindex="101" value="User Name" onfocus="if (this.value == 'User Name') this.value = '';" /></td>
			<td class="smallfont" colspan="2" nowrap="nowrap"><label for="cb_cookieuser_navbar"><input type="checkbox" name="cookieuser" value="1" tabindex="103" id="cb_cookieuser_navbar" accesskey="c" />Remember Me?</label></td>
		</tr>
		<tr>
			<td class="smallfont"><label for="navbar_password">Password</label></td>

			<td><input type="password" class="bginput" style="font-size: 11px" name="vb_login_password" id="navbar_password" size="10" tabindex="102" /></td>
			<td><input type="submit" class="button" value="Log in" tabindex="104" title="Enter your username and password in the boxes provided to login, or click the 'register' button to create a profile for yourself." accesskey="s" /></td>
		</tr>
		</table>
		<input type="hidden" name="s" value="" />
		<input type="hidden" name="do" value="login" />		
		<input type="hidden" name="vb_login_md5password" />
		<input type="hidden" name="vb_login_md5password_utf" />
		</form>

		<!-- / login form -->

whic is a bit more involved than the simple login stuff you are sending to the server.

- Kevin, perl coder unexceptional!

 

[titosat]

Is working now Someone verify if is all right and how i do to make the script to run every 20 sec and like a timer or somethin. and is not to musch to ask pls help me to add to the script to connect to mysql and send the data like

INSERT INTO `database_name` (`link_author`,`link_category`,`link_title`,`link_url`,`link_title_url`,`link_content`,`link_url_title`,`link_tags`,`link_status`,`link_published_date`) VALUES ('...');

use strict;
use warnings;
use HTTP::Cookies;
use LWP::UserAgent;
use HTTP::Request::Common qw(POST);
use HTTP::Request::Common qw(GET);

my $ua = new LWP::UserAgent;
my $cookie_jar = HTTP::Cookies->new(file => 'lwpcookies.txt',
  autosave => 1, ignore_discard => 1);
$ua->cookie_jar($cookie_jar);
my $user = "user";
my $pass = "password";
my $url = '[URL unfurl="true"]http://www.totalfta.com/login.php';[/URL]
my $url2 = '[URL unfurl="true"]http://www.totalfta.com/search.php?do=getnew';[/URL]

my $req = POST $url, Content_Type => 'form-data', Content => [ 'do' => 'login', vb_login_username => $user, vb_login_password => $pass ];
my $res = $ua->request($req);

my $req2 = GET $url2;
my $res2 = $ua->request($req2); 


my $content = $res2->as_string;
 if ( $content =~ /<td class="alt1" id="(.+)" title="(.+)">/m) {
  print "thread $2\n";     
 }
  if ( $content =~ /<a href="(.+)" id="(.+)" style="font-weight:bold">(.+)<\/a>/m ) {
     print "thread title: $2\n";     
 }
 if ( $content =~ /<td class="alt2" title="Replies:(.+), Views:(.+)">/m ) {
    print "Replies: $1, Views: $2\n";     
 }
 
 if ( $content =~ /by <a href="(.+)" rel="nofollow">(.+)<\/a>  <a href="(.+)"><img class="inlineimg" src="(.+)" alt="Go to last post" border="0" \/><\/a>/m ) {
    print "by: $2\n";     
 }

 if ( $content =~ /<td class="alt1"><a href="(.+)">(.+)<\/a><\/td>/m) {
  print "Forum Section: $2\n";     
 }