×
INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

extracting values from multiple lines into a seperate output file

extracting values from multiple lines into a seperate output file

extracting values from multiple lines into a seperate output file

(OP)
Hi,

I’m using perl on windows v5.10.1 and would like to extract out of a log file extract below to produce output as below, does anyone know how to extract the items highlighted in red? The format of the log file is pretty much consistent throughout:

Log extract

CCurClgA=901922494724 MetaCode(true)=META_UNKNOWN'
11/07/12 15:49:05.448 Trc 04541 RequestReleaseCall received from [8220] (0001238c 123103 172.46.713.123:1966)
message RequestReleaseCall
AttributeReferenceID 219
AttributeConnID 00c10218203ee739
AttributeThisDN '82123345'
11/07/12 15:49:05.448 Int 04543 Interaction message "RequestReleaseCall" received from 8220 ("123103")
@15:49:05.4480 [gctmi] Distributing request RequestReleaseCall
@15:49:05.4480 [tout] Party [00c10218208ee739:89123103,sc,tDN,rDST,lINT] (reqReleaseCall)

Output I’d like to pull from log file

Date Time Event IP ConnID DN
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345


Any help would be appreciated greatly 

Thanks

RE: extracting values from multiple lines into a seperate output file

What have you tried? What are you having problems with?

RE: extracting values from multiple lines into a seperate output file

(OP)
This is what I have so far, the bit I struggle with mostly is extracting the lines below my initial line to get the values I'm interested in I think there's also some issue with my syntax?

use strict;
use warnings;

print " Date\tTime\tEvent\tIP Address\tConnID\tDN\n ";

@ARGV = <@ARGV>;

while (<>) {

if
(/^([\d\/]+)\s([\d:\.]+)\[Trc\]\s\d+\s(RequestReleaseCall)\s[recieved]\s[from]\s\[d+\]\s\(:\d+c\s\d+\s([^\:]+)/){print "$1 $2 $3 $4 \n"}

}


Cheers!

RE: extracting values from multiple lines into a seperate output file

Hi

Yes, beside some missing and extra characters there are also some pointless escapes :

CODE --> regular-expression

# before
/^([\d\/]+)\s([\d:\.]+)\[Trc\]\s\d+\s(RequestReleaseCall)\s[recieved]\s[from]\s\[d+\]\s\(:\d+c\s\d+\s([^\:]+)/

# after
/^([\d\/]+)\s([\d:.]+)\sTrc\s\d+\s(RequestReleaseCall)\sreceived\sfrom\s\[\d+\]\s\(\d+c\s\d+\s([^:]+)/ 

Regarding extracting pieces from the other lines too, just add more similar regular expressions and prints. Just spare the \n from the end of printed strings, excepting the last one.

Feherke.
http://feherke.github.com/

RE: extracting values from multiple lines into a seperate output file

(OP)
Thanks Ferkhe,

"Regarding extracting pieces from the other lines too, just add more similar regular expressions and prints. Just spare the \n from the end of printed strings, excepting the last one."

My log file has many instances of the word AttributeConnID and AttributeThis DN, it is only when they occur 3 and 4 lines below the word RequestReleaseCall is when I'm interested in extracted will the above work in this case? Will it not pull all the other matches that aren't 3&4 lines below RequestReleaseCall keyword

11/07/12 15:49:05.448 Trc 04541 RequestReleaseCall received from [8220] (0001238c 123103 172.46.713.123:1966)
message RequestReleaseCall
AttributeReferenceID 219
AttributeConnID 00c10218203ee739 <------- 3 lines below RequestReleaseCall
AttributeThisDN '82123345' <--------4 lines below RequestReleaseCall
11/07/12 15:49:05.448 Int 04543 Interaction message "RequestReleaseCall" received from 8220 ("123103")
@15:49:05.4480 [gctmi] Distributing request RequestReleaseCall

Thanks

RE: extracting values from multiple lines into a seperate output file

Hi

Well, the information about repeated AttributeConnID and AttributeThisDN lines was missing in your original post. Further unspecified details :
  • Is there a single RequestReleaseCall line or can be more ?
  • If there are more RequestReleaseCall lines, only the first has to be extracted or all ?
  • Are the lines always in RequestReleaseCall, AttributeConnID, AttributeThisDN order ?
  • Are there always AttributeConnID and AttributeThisDN lines at 3-4 line distance after the RequestReleaseCall lines ?
If you want check the distance between the matched lines, use a counter : initialize it when RequestReleaseCall line is found, increment it on every line, check it when AttributeConnID and AttributeThisDN lines are found.

Feherke.
http://feherke.github.com/

RE: extracting values from multiple lines into a seperate output file

(OP)
Thanks

I do seem to have left some key details out apologies

Is there a single RequestReleaseCall line or can be more ? <---- Single on a line but many instances which I'd like to extract
If there are more RequestReleaseCall lines, only the first has to be extracted or all ? <--- All

Are the lines always in RequestReleaseCall, AttributeConnID, AttributeThisDN order ? <--- Yes
Are there always AttributeConnID and AttributeThisDN lines at 3-4 line distance after the RequestReleaseCall lines ? <-- Yes

Thanks

RE: extracting values from multiple lines into a seperate output file

Hi

Well, if you can always count on the presence of each line, then you can do it without counting the lines. Just collect the needed pieces and output them when all 6 found :

CODE --> Perl

use strict;
use warnings;

print "Date\tTime\tEvent\tIP Address\tConnID\tDN\n";

@ARGV = <@ARGV>;

my @out;
while (<>) {
  @out = ($1,$2,$3,$4) if /^([\d\/]+) ([\d:.]+) Trc \d+ (RequestReleaseCall) received from \[\d+\] \(\d+c \d+ ([^:]+)/;
  push @out, $1 if /^AttributeConnID (.+)/;
  push @out, $1 if /^AttributeThisDN '(.+)'/;

  print join("\t", @out), "\n" if scalar @out == 6;
} 

( Note that using \s instead of space makes the regular expression harder to read. We used to use \s in our responses just in case the tab and space characters were messed up on their way through the forum's web interface when posted. If you are sure the words in the parsed text are always separated by spaces, better write just spaces in the regular expression. )

Feherke.
http://feherke.github.com/

RE: extracting values from multiple lines into a seperate output file

(OP)
Thanks that's brilliant almost gives it what I need, one thing though for each result found it's returning the row 6 times? so for the extract of the log file above it returns the following, the same line repeated 6 times?

Date Time Event IP Address ConnID DN
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345
11/07/12 15:49:05.448 RequestReleaseCall 172.46.713.123 00c10218203ee739 82123345

Out of interest how does the push @out work?

push @out, $1 if /^AttributeConnID (.+)/;


Cheers

RE: extracting values from multiple lines into a seperate output file

Hi

Quote (mmiah1982)

or each result found it's returning the row 6 times?
Oops. Seems my test input file was too simple. A quick fix, I hope :

CODE --> Perl

use strict;
use warnings;

print "Date\tTime\tEvent\tIP Address\tConnID\tDN\n";

@ARGV = <@ARGV>;

my @out;
while (<>) {
  @out = ($1,$2,$3,$4) if /^([\d\/]+) ([\d:.]+) Trc \d+ (RequestReleaseCall) received from \[\d+\] \(\d+c \d+ ([^:]+)/;
  push @out, $1 if /^AttributeConnID (.+)/;
  if (/^AttributeThisDN '(.+)'/) {
    push @out, $1;
    print join("\t", @out), "\n" if scalar @out == 6;
  }
} 

Quote (mmiah1982)

Out of interest how does the push @out work?
push appends the specified value(s) ( 2nd and following parameters ) to the end of given array ( 1st parameter ).

Feherke.
http://feherke.github.com/

RE: extracting values from multiple lines into a seperate output file

(OP)
Thats done the trick! Brill Thanks for your help :)

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members! Already a Member? Login

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close