Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Register Log in
  • Congratulations wOOdy-Soft on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Perl file parsing and extracting certain info

Status
Not open for further replies.
dietmarp

dietmarp

Programmer
Dec 27, 2001
53
US
Hi list - how would I do following.

I've got a file with over 500 blocks like following. I need to parse the file consisting of blocks starting with CLASS end Ending with END. Inside those block is the information I need to extract. For all lines lines inside each block starting with 1: to 11: I need to extract the values inside the " " and put it in one line into a new file.
I did it in a shell script and the script ran for 4,5 hours. So I like to rewrite in perl.


CLASS IM_15004
SELECT
1: ATTR(=,$ENTERPRISE) , VALUE PREFIX, "1.3.6.1.4.1.232" ) ;
2: $SPECIFIC = 15004;
3: ATTR(=, "sysName") ;
4: ATTR(=, "cpqClusterNodeName");
MAP
sub_source = HP_Insight_Manager;
specificTrap = $V2;
severity = FATAL;
hostname=$V3;
hpq_ClusterNodeName = $V4;
hpq_eventID = $V2;
msg = "Cluster Node Failed";
hpq_info = PRINTF ( "Cluster service on %s has failed.", $V4);
END

CLASS IM_15005
SELECT
1: ATTR(=,$ENTERPRISE) , VALUE(PREFIX, "1.3.6.1.4.1.232" ) ;
2: $SPECIFIC = 15005;
3: ATTR(=, "sysName") ;
4: ATTR(=, "cpqClusterResourceName");
MAP
sub_source = HP_Insight_Manager;
specificTrap = $V2;
severity = CRITICAL;
hostname=$V3;
hpq_ClusterResourceName = $V4;
hpq_eventID = $V2;
msg = "Cluster Resource Degraded";
hpq_info = PRINTF ( "Cluster resource %s has become degraded.", $V4);
END
 
Sort by date Sort by votes
Hi dietmarp

I'm not sure i quite understand - but how is this for a start?

Code: 
[b]#!/usr/bin/perl[/b]

while (<DATA>) {
  print "$1\n" if /\b\d+: .* "([^"]+)"/;
}

[blue]__DATA__
CLASS IM_15004
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15004;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterNodeName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = FATAL;
    hostname=$V3;
    hpq_ClusterNodeName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Node Failed";
    hpq_info = PRINTF ( "Cluster service on %s has failed.", $V4);
END

CLASS IM_15005
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE(PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15005;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterResourceName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = CRITICAL;
    hostname=$V3;
    hpq_ClusterResourceName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Resource Degraded";
    hpq_info = PRINTF ( "Cluster resource %s has become degraded.", $V4);
END[/blue]

outputs:-

[red]1.3.6.1.4.1.232
sysName
cpqClusterNodeName
1.3.6.1.4.1.232
sysName
cpqClusterResourceName[/red]


Kind Regards
Duncan
 
Upvote 0 Downvote
A quick simple example for you.
You did not specify the output very precisely so you may have to tinker this:
Code: 
#!/usr/bin/perl -w
use strict;
my @strings;
while(<DATA>) {
  chomp;
  @strings = () if(/^CLASS/);
  push @strings, $1 if(/^\s+\d+:[^"]+("[^"]+")/);
  print join(",", @strings), "\n" if(/^END/);
}
__DATA__
CLASS IM_15004
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15004;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterNodeName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = FATAL;
    hostname=$V3;
    hpq_ClusterNodeName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Node Failed";
    hpq_info = PRINTF ( "Cluster service on %s has failed.", $V4);
END

CLASS IM_15005
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE(PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15005;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterResourceName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = CRITICAL;
    hostname=$V3;
    hpq_ClusterResourceName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Resource Degraded";
    hpq_info = PRINTF ( "Cluster resource %s has become degraded.", $V4);
END



Trojan.
 
Upvote 0 Downvote
hehehe
Great minds think alike!
Mine prints it as comma separated on the same line.
I think you need to spec your output more carefully.
And to run, just loose everything after (and including) the __DATA__ line and remove the DATA from the <DATA>.



Trojan.
 
Upvote 0 Downvote
Your well on the way to a 200% ratio now! The counters are on your side!!!


Kind Regards
Duncan
 
Upvote 0 Downvote
Hi, thanks for your input so far.

Thats, how the output should look like.

IM_15004
1.3.6.1.4.1.232 15004 sysName cpqClusterNodeName
IM_15005
1.3.6.1.4.1.232 15005 sysName cpqClusterResourceName
....
...
 
Upvote 0 Downvote
Try this then:
Code: 
#!/usr/bin/perl -w
use strict;
my @strings;
while(<DATA>) {
  chomp;
  @strings = (), print "$1\n" if(/^CLASS (.*)$/);
  push @strings, $1 if(/^\s+\d+:[^"]+"([^"]+)"/);
  print join(" ", @strings), "\n" if(/^END/);
}
__DATA__
CLASS IM_15004
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15004;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterNodeName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = FATAL;
    hostname=$V3;
    hpq_ClusterNodeName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Node Failed";
    hpq_info = PRINTF ( "Cluster service on %s has failed.", $V4);
END

CLASS IM_15005
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE(PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15005;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterResourceName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = CRITICAL;
    hostname=$V3;
    hpq_ClusterResourceName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Resource Degraded";
    hpq_info = PRINTF ( "Cluster resource %s has become degraded.", $V4);
END

Output is:
IM_15004
1.3.6.1.4.1.232 sysName cpqClusterNodeName
IM_15005
1.3.6.1.4.1.232 sysName cpqClusterResourceName



Trojan.
 
Upvote 0 Downvote
Hi Trojan, that comes very close, but 15005 isn't in the output yet. This is the only line differning, since the value is not in "", but it always follows the $SPECIFIC =
Dietmar
 
Upvote 0 Downvote
Code: 
#!/usr/bin/perl -w
use strict;
my @strings;
while(<DATA>) {
  chomp;
  @strings = (), print "$1\n" if(/^CLASS (.*)$/);
  push @strings, $1 if(/^\s+\d+:[^"]+"([^"]+)"/);
  push @strings, $1 if(/^\s+\d+:\s+\$SPECIFIC\s+=\s+(\d+);/);
  print join(" ", @strings), "\n" if(/^END/);
}
__DATA__
CLASS IM_15004
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15004;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterNodeName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = FATAL;
    hostname=$V3;
    hpq_ClusterNodeName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Node Failed";
    hpq_info = PRINTF ( "Cluster service on %s has failed.", $V4);
END

CLASS IM_15005
  SELECT
    1: ATTR(=,$ENTERPRISE) , VALUE(PREFIX, "1.3.6.1.4.1.232" ) ;
    2: $SPECIFIC = 15005;
    3: ATTR(=, "sysName") ;
    4: ATTR(=, "cpqClusterResourceName");
  MAP
    sub_source = HP_Insight_Manager;
    specificTrap = $V2;
    severity = CRITICAL;
    hostname=$V3;
    hpq_ClusterResourceName = $V4;
    hpq_eventID = $V2;
    msg = "Cluster Resource Degraded";
    hpq_info = PRINTF ( "Cluster resource %s has become degraded.", $V4);
END


Trojan.
 
Upvote 0 Downvote
I put a specific for $SPECIFIC (if you see what I mean) which is probably overkill, I could have just removed the quotes but you did specify quotes in your first post so I'm assuming that you don't want anything else that's not in quotes.
Either way, this should do what you want.

As before, remove the DATA from <DATA> and run against your files to test.



Trojan.
 
Upvote 0 Downvote
Hi, thanks a lot - it works - sooner or later I'll be able to do the regex myself.
One more question:
From the output i.e. 1.3.6.1.4.1.232 15004 sysName cpqClusterNodeName
I'd like to check (in shell I used a grep $trap $file|awk -F'{print ..)
if this pattern exist and then replace it with the found string.
Example of trap.oid file
"cpqClusterNodeName" "1.3.6.1.4.1.232.15.2.3.1.1.2"

So the final result would look like this:

1.3.6.1.4.1.232 15004 sysName 1.3.6.1.4.1.232.15.2.3.1.1.
 
Upvote 0 Downvote
ups - I ment to say if this pattern exist in another file ie trap.oid where to perform the grep
 
Upvote 0 Downvote
I'm sorry, I didn't really understand your explanation of what you want to do next.
Where are you looking up the cpqClusterNodeName?
How do you convert it to OID?


Trojan.
 
Upvote 0 Downvote
Code: 
grep cpqClusterNodeName *.oid | awk '{print $2}'

... then remove the speech marks

this sort of thing?


Kind Regards
Duncan
 
Upvote 0 Downvote
If the data is in the file we are processing then we can do it all in one hit.
I just need to know what you want.



Trojan.
 
Upvote 0 Downvote
Ups - ok. I've got two files. A trap.cds and a trap.oid
trap.cds containing the snmp trap definitons
i.e.
CLASS IM_153001_2
SELECT
1: ATTR(=,$ENTERPRISE) , VALUE(PREFIX, "1.3.6.1.4.1.232.153" ) ;
2: $SPECIFIC = 153001;
3: ATTR(=, "sysDescr") ;
4: ATTR(=, "sysContact");
5: ATTR(=, "sysName");
6: ATTR(=, "sysLocation");
7: ATTR(PREFIX, "cpqCmcStatusTemp1"), VALUE(=,2);
MAP
sub_source = HP_Insight_Manager;
specificTrap = $V2;
hpq_eventID = 153001_2;
severity = HARMLESS;
hpq_contact = $V4;
hostname=$V5;
hpq_location = $V6;
hpq_CmcStatusTemp1 = $V7;
sub_origin = $K7;
hpq_val = $V7;
msg = "The status of Compaq ConsoleManagementController temperature 1: Normal";
hpq_info = PRINTF ( "Location: %s, Message: %s from: %s, contact %s", $V6, $V7, $V5, $V4);
END

and then trap.oid file containing the snmp id's.

i.e.
"cpqCmcStatusTemp1" "1.3.6.1.4.1.232.153.2.2.4.1"
"cpqCmcStatusTemp2" "1.3.6.1.4.1.232.153.2.2.4.2"
"sysDescr" "1.3.6.1.2.1.1.1"
"sysObjectID" "1.3.6.1.2.1.1.2"
"sysContact" "1.3.6.1.2.1.1.4"
"sysName" "1.3.6.1.2.1.1.5"
"sysLocation" "1.3.6.1.2.1.1.6"
"ssSubsysName" "1.3.6.1.4.1.36.2.15.21.1.3.2.1.2"
"cpqSwccKzpccSubsystemName" "1.3.6.1.4.1.232.132.4.1.2"
"cpqClusterNodeName" "1.3.6.1.4.1.232.15.2.3.1.1.2"

So for every string we extracted before I need to grep the corresponding oid and replace it.
So in this example using your scriptlet the result would be:

1.3.6.1.4.1.232.1 153001 sysDescr sysContact sysName sysLocation cpqCmcStatusTemp1

and after it should be

1.3.6.1.4.1.232.1 153001 1.3.6.1.2.1.1.1 1.3.6.1.2.1.1.4 1.3.6.1.2.1.1.5 1.3.6.1.2.1.1.6 1.3.6.1.4.1.232.153.2.2.4.1

Thats the information I need to simulate SNMP Traps of HP Compaq Hardware.
Sorry for the misunderstanding description before -;)
 
Upvote 0 Downvote
Hi dietmarp

where do you get 1.3.6.1.4.1.232.1 from?

1.3.6.1.4.1.232.1 153001 sysDescr sysContact sysName sysLocation cpqCmcStatusTemp1


Kind Regards
Duncan
 
Upvote 0 Downvote
Try this:

Code: 
#!/usr/bin/perl -w
use strict;
my %hash = ();
open FH, "trap.oid" or die "Failed to open trap.oid";
while(<FH>) {
  /"([^"]+)"\s+"([^"]+)"/
    and $hash{$1} = $2;
}
close FH;
open FH, "trap.cds" or die "Failed to open trap.cds";
my @strings;
while(<FH>) {
  chomp;
  @strings = (), print "$1\n" if(/^CLASS (.*)$/);
  if(/^\s+(\d+):[^"]+"([^"]+)"/) {
    if($1 == 4 and exists $hash{$2}) {
      push @strings, $hash{$2};
    } else {
      push @strings, $2;
    }
  }
  push @strings, $1 if(/^\s+\d+:\s+\$SPECIFIC\s+=\s+(\d+);/);
  print join(" ", @strings), "\n" if(/^END/);
}
close FH;

Trojan.
 
Upvote 0 Downvote
Status
Not open for further replies.

Similar threads

rob54321
  • Locked
  • Question
Perl how to read line from file with interpolation 1
Replies
6
Views
991
rob54321
rob54321
newbie1238585
  • Locked
  • Question
Perl how to read string in csv file * as wildcard operator
Replies
2
Views
590
mikrom
mikrom
Limbomusic
  • Locked
  • Question
Perl files getting accessed directly
Replies
0
Views
441
Limbomusic
Limbomusic
ss_ham
  • Locked
  • Question
in Perl find call looking to exclude folder and ignore duplicate finds.
Replies
2
Views
495
ss_ham
ss_ham
eslie
  • Locked
  • Question
Perl script behavior makes no sense!!!
Replies
0
Views
370
eslie
eslie

Part and Inventory Search

Sponsor

Back
Top