×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

PCAP reader to mysql

PCAP reader to mysql

PCAP reader to mysql

(OP)
Hi i am just learning perl for a company project they want me to make a pcap reader that puts the information into mysql.

I looked into what i would need and realized that tcpdumplog would be good for reading the pcap file but i am not sure where to go with the code to get it to save in record format to mysql.

So correct me if im not on the right track but this is what I was working with!

#!/usr/bin/perl

use Net::TcpDumpLog;

my $log = Net::TcpDumpLog->new();
$log->read("C:\Documents and Settings\jordant\My Documents\pcap\m1.pcap");


I still get an error with the above but wont that read my m1.pcap file? How do i get it to mysql format?

Any help is appreciated!

RE: PCAP reader to mysql

(OP)
Ok so i did some more research and found a cleaner code with more lables for easier identifcation.

#!/usr/bin/perl

use Net::TcpDumpLog;
use NetPacket::Ethernet;
use NetPacket::IP;
use NetPacket::TCP;
use strict;
use warnings;

my $log = Net::TcpDumpLog->new();
$log->read("/Dump/m1.pcap");

foreach my $index ($log->indexes) {
  my ($length_orig, $length_incl, $drops, $secs, $msecs) = $log->header($index);
  my $data = $log->data($index);

  my $eth_obj = NetPacket::Ethernet->decode($data);
  next unless $eth_obj->{type} == NetPacket::Ethernet::ETH_TYPE_IP;

  my $ip_obj = NetPacket::IP->decode($eth_obj->{data});
  next unless $ip_obj->{proto} == NetPacket::IP::IP_PROTO_TCP;

  my $tcp_obj = NetPacket::TCP->decode($ip_obj->{data});
  my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($secs + $msecs/1000);
  print sprintf("%02d-%02d %02d:%02d:%02d.%d",
    $mon, $mday, $hour, $min, $sec, $msecs),
    " ", $eth_obj->{src_mac}, " -> ",
    $eth_obj->{dest_mac}, "\n";
  print "\t", $ip_obj->{src_ip}, ":", $tcp_obj->{src_port},
    " -> ",
    $ip_obj->{dest_ip}, ":", $tcp_obj->{dest_port}, "\n";
}


My issue is $log->read. seems to always not find my file. I have my file on the Desktop. I tried using a direct path to it but maybe perl needs a specific way? Sorry im new.

Also how can i get this to database into mysql?? Thanks again

RE: PCAP reader to mysql

$log->read("/Dump/m1.pcap");
 does not look like it is on your desktop.  You will have to deal with the spaces in windows files name, I suggest just keeping in a directory with no no spaces c:/blah/m1.pcap and reading it in like that "c:/blah/m1.pcap".
 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
noevil
Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;

RE: PCAP reader to mysql

(OP)
Ive got the Pcap reading out because i can use the print command to see it. But how would i get it into mysql?

#Login to mysql
$dbh = DBI->connect('DBI:mysql:test', 'root', 'password'
               ) || die "Could not connect to +database: $DBI::errstr";

 #The Headers in Table
$dbh->do("CREATE TABLE test2 (Source   , Destination VARCHAR(100), Packets VARCHAR(100))");

  #Info in Table
$dbh->do( "INSERT INTO test2 (Source,Destination,Packets)
                        values ('$ip_obj','$tcp_obj','$eth_obj')");

So my question is the last part how would i produce my earlier tcpdump into the table???

Also how do you get TCPDUMPLOG to show the packets??

RE: PCAP reader to mysql

(OP)
2 things left to do on it.

Show output from pcap of time when packets were used.
Show output for Packets being used (in bytes)
 

RE: PCAP reader to mysql

(OP)
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($secs + $msecs/1000);


Thats my time string but it seems to not work correctly.

I get the right month and day. But year and Mins/Sec are off.

RE: PCAP reader to mysql

How far off?  And for what values of $secs and $msecs?

Annihilannic
tgmlify - code syntax highlighting for your tek-tips posts

RE: PCAP reader to mysql

(OP)
Year is way off. sometimes gives the ecop 1969. Do you have a code to output time from the pcap when packets were taken?

I went another path and getting wrong dates as well.


use Time::HiRes qw(gettimeofday);  

    @HiResTime = gettimeofday();
    ($sec,$min,$hour,$day,$month,$year) = localtime($HiResTime[0]);


 $month = $month + 1;
$year = $year + 1900;
my $time = sprintf "%02d/%02d/%4d %02d:%02d:%02d:%06d",
$month,$day,$year,$hour,$min,$sec,$HiResTime[1];

outputs: 111/05/211012   16:22:19:578125

Times are all the same and 111 goes up to 172.

05 all the same. 3rd number is 6 digits  in 300,000 range.

 

RE: PCAP reader to mysql

Please output the values of $secs and $msecs from your original code and let us know what exact (incorrect) values are returned by localtime().

For your second example, you haven't provided a complete list of values on the left hand side of localtime().  You can't just omit the ones you don't want because localtime() returns all of them anyway; by doing so you will end up with unexpected values in your variables.  You can use undef placeholders if you'd rather not define variables for the unused values.

I don't think localtime() understands the HiResTime value you are supplying anyway... not sure about that?

Annihilannic
tgmlify - code syntax highlighting for your tek-tips posts

RE: PCAP reader to mysql

(OP)
Is there another value other then localtime to be used? I added all the variables for local time and the $month comes back as 1-172 counting up by 1. because of $month = $month + 1;

my year is insanley big 300,000. My time is just my current computer time.


I need something to grab the times from the pcap. Wireshark has it in Seconds. I cant even get that output in perl.

RE: PCAP reader to mysql

Can you show us a sample of the timestamps returned by pcap?  I don't have it installed so I have no idea what it looks like.

Annihilannic
tgmlify - code syntax highlighting for your tek-tips posts

RE: PCAP reader to mysql

(OP)
Ive got it working properly now. But here is my final question. If i have it reading a log file like so:

$log->read("C:\\Documents and Settings\\jordant\\Desktop\\Dump\\m1.pcap")

How could i get it so it reads within the Dump folder but all files that are .pcap?

i will run into problems putting each file in individually because we will be adding more i want it to read all files(that are pcap files)within the folder [C:\\Documents and Settings\\jordant\\Desktop\\Dump].

RE: PCAP reader to mysql

Wrap a loop like this around it:

CODE --> Perl

#!/usr/bin/perl -w
use strict;

foreach my $pcapfile (glob "C:\\Documents and Settings\\jordant\\Desktop\\Dump\\*.txt") {
        $log->read($pcapfile)
        # ...
}

Annihilannic
tgmlify - code syntax highlighting for your tek-tips posts

RE: PCAP reader to mysql

(OP)
Alright so i tryed adding this into my code but for some reason i get an error.

Here is the exact message:
ERROR: Can't read log C:./Documents: No such file or directory.

The directory does exist tho. I can even put back the exact path with it going to that directory to m1.pcap and it works...?

Here is my entire code.

CODE

#!/usr/bin/perl -w


 use DBI;
 use Net::TcpDumpLog;
 use NetPacket::Ethernet;
 use NetPacket::IP;
 use NetPacket::TCP;
 use Net::Pcap;


 #Login to mysql
$dbh = DBI->connect('DBI:mysql:test', 'root', 'nstar'
               ) || die "Could not connect to +database: $DBI::errstr";



 #Pcap file to log
 my $log = Net::TcpDumpLog->new();
 foreach my $pcapfile (glob "C:\\Documents and Settings\\jordant\\Desktop\\Dump\\*.pcap") {
        $log->read($pcapfile)
        }

 #INFO from PCAP file
foreach my $index ($log->indexes) {
  my ($length_orig, $length_incl, $drops, $secs, $msecs) = $log->header($index);
  my $data = $log->data($index);



  my $eth_obj = NetPacket::Ethernet->decode($data);
  next unless $eth_obj->{type} == NetPacket::Ethernet::ETH_TYPE_IP;


  my $ip_obj = NetPacket::IP->decode($eth_obj->{data});
  next unless $ip_obj->{proto} == NetPacket::IP::IP_PROTO_TCP;

  my $tcp_obj = NetPacket::TCP->decode($ip_obj->{data});





      #get date time stamp of packet
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($secs + $msecs/1000);
    $mon+=1;
  my $time = sprintf("%02d-%02d %02d:%02d:%02d",
    $mon, $mday, $hour, $min, $sec);





  #Info in Table

  $dbh->do( "INSERT INTO  test2 (Date,Source,Destination,Packets,Port)
                        values (
                        '$time',
                        '$ip_obj->{src_ip}',
                        '$ip_obj->{dest_ip}',
                        '$ip_obj->{len}',
                        '$tcp_obj->{dest_port}')");


  }

RE: PCAP reader to mysql

If you add a print message like this, does it look like it is trying to read the correct file names?

CODE

        print "Reading $pcapfile"
        $log->read($pcapfile)
 

Annihilannic
tgmlify - code syntax highlighting for your tek-tips posts

RE: PCAP reader to mysql

Sorry can't provide any help but I like the idea of what you are doing. When you get it all worked out can you post the complete code?? Thanks

RE: PCAP reader to mysql

(OP)
No Annihilannic. It errors out with no directory/or file like i said above. I worked on it for a little and i got it outputting cannont read m2.pcap because its not a file.?

Here what i changed it too.

CODE

    my $dir = 'C:\\Documents and Settings\\jordant\\Desktop\\Dump\\';

    opendir(DIR, $dir) or die $!;

    while (my $file = readdir(DIR)) {

        # We only want files
        next unless (-f "$dir/$file");

        # Use a regular expression to find files ending in .pcap
        next unless ($file =~ m/\.pcap$/);

     my $log = Net::TcpDumpLog->new();
      $log->read($file)
      }
       print $file;
      exit 0;

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close