Regex help please 1

stevio · Feb 16, 2009

Have some data like the following

Product : CCS
SerialNumber : 75010010
LUs : 2

iLU HDevName OSPathID PathID ChaPort Status Type
0026 hdisk24 00000 000018 1A Online Non
00001 000019 0A Online Own
0148 hdisk29 00000 000020 1A Online Non
00001 000021 0A Online Own

Product : SIP
SerialNumber : 0010111
LUs : 3

iLU HDevName OSPathID PathID ChaPort Status Type
0010 hdisk21 00000 000040 2A Online Own
00001 000041 1A Online Own
0020 hdisk26 00000 000017 1A Online Own
0021 hdisk22 00000 000039 2A Online Own

Product : SIP
...etc

KAPL01001 # ends here

Serial number can either be 8 digits(product CCS) or 7 digits (product SIP) and it is in no particular order

Trying to regex to do 2 things

1) if product is CCS (serial is 8 digits), truncate leading zeros and print

2) if there is a line like so (for other product)

0026 hdisk24 00000 000018 1A Online Non
00001 000019 0A Online Own

then print them together, because they belong together i.e. always grab the following line if we find the first one.

So far I've tried this, and I am getting so frustrated

Code:

my $data_file = "myfile.txt";

open DATA, "$data_file" or die "can't open $data_file $!";

while(<DATA>)
{
    if (/^Product\s+:\s+CCS/ ... /^Product\s+:   \s+(?:CCS|PIS|^KAP)/){ 
      if (/^(\w{4})\s+\w+\s+.*/) {
         my $lu = $1;
         my $lu =~ s/^0+//;
         print "CCS $lu\n";
      }
    }

    elsif (/^Product\s+:\s+PIS/ ... /(?:^KAPL)/){
         print "Start of block\n";
         if (/^\w{4}\s+\w+\s+.*/) {
             print "PIS $_\n";
         }
    }

}

Any help appreciated.

KevinADC · Feb 16, 2009

I don't understand what you are trying to do with the data file. Maybe a before and after example will help. Put code and data in code tags please.

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]

stevio · Feb 16, 2009

What I eventually want to do is to put each product into it's own hash. I figure that if I could at least separate them out, then I could put them into a hash

I want to achieve 2 things

1) if product is CCS, then truncate leading zeros off the iLU column,

so

Code:

0026 hdisk24  00000    000018 1A      Online     Non

0026 becomes after capture

becomes after stripping off leading zeros

26

2) For each line which has the following format

Code:

0026 hdisk24  00000    000018 1A      Online     Non
              00001    000019 0A      Online     Own

If the line has this format, then they both of them need to be inserted into the hash - see below.

in the one case

Code:

0021 hdisk22  00000    000039 2A      Online     Own

there is no second line

So final hash/array might have the following structure

Code:

75010010
iLU   Port Type
26  => 1A => Non
    => 0A => Own
148 => 1A => Non
    => 0A => Own

0010111
iLU   Port Type
0010 =>2A => Own
     =>1A =>Own
...
0021 =>2A => Own 

etc

I hope this makes more sense.

ejaggers · Feb 17, 2009

How about something like this:

open(IFH,"tt.INP");
my @input = <IFH>;
my %hash;
my ($product,$serial,$iLUfound);
foreach ( @input ) {
chomp;
next if ( /^\s*$/ );
if ( /^product/i ) {
($product,$product) = split(' : ',$_);
$iLUfound = '';
next;
}
if ( /^serial/i ) {
($serial,$serial) = split(' : ',$_);
next;
}
$iLUfound = 'yes' if ( /^iLU/i );
next unless $iLUfound;

if ( $product eq 'CCS' ) {
next unless /^\d/;
s/^0+//;
my ($iLU) = split;
$hash{"$product,$serial,$iLU"} = $_;
}
if ( $product ne 'CCS' ) {
if ( /^\w/ ) {
my ($iLU) = split;
$Key = "$product,$serial,$iLU";
$hash{$Key} = $_;
next;
}
if ( $hash{$Key} ) {
$hash{$Key} .= $_;
undef $key;
}
}
}
foreach ( sort keys %hash ) {
print "product,serial,iLu = $_\n$hash{$_}\n\n";

}

stevio · Feb 17, 2009

Thanks ejaggers, goes a long way to solving the problem (and stress)

Quick question

Can you pls explain this section of code, esp the line

$hash{$Key} . = $_; # what is the '.' do?

Code:

if ( $hash{$Key} ) {
                  $hash{$Key} .= $_;
                  undef $key; #undefined key..?
             }

BTW, have a star and thanks for spending the time to help.

KevinADC · Feb 17, 2009

My apologies for getting back so late. Here is the code I wrote that produces only the port and type data associataed with the serial number

Code:

use warnings;
use strict;
use Data::Dumper;
my %hash;
open(my $IN, "<", 'c:/perl_test/test.txt') or die "$!";
OUTTER: while(<$IN>){
   my ($prod,$ser,$LU);
   chomp;
   if (/^Product\s+:\s+(CCS|SIP)/) {
      $prod = $1;
      ($ser,$LU) = (scalar <$IN>, scalar <$IN>);
      tr/0-9//cd for ($ser,$LU);
      <$IN>,<$IN>;#read next two lines and discard
   }
   my ($ilu,$port,$type);
   INNER: while (my $line = <$IN>) {
      chomp($line);
      if ($line =~ /^\d/) {
# parse lines like: 0026 hdisk24  00000    000018 1A      Online     Non
         ($ilu,$port,$type) = (split(/\s+/,$line))[0,4,6];
         $ilu = int($ilu) if ($prod eq 'SIP');
         push @{$hash{$ser}{$ilu}{Port}},$port;
         push @{$hash{$ser}{$ilu}{Type}},$type;
         next INNER;
      }
      elsif ($line =~ /^\s+\d/) {
# parse lines like:               00001    000019 0A      Online     Own
         ($port,$type) = (split(/\s+/,$line))[3,5];
         push @{$hash{$ser}{$ilu}{Port}},$port;
         push @{$hash{$ser}{$ilu}{Type}},$type;
         next INNER;
      }
      else {
         next OUTTER;
      }
   }
}
print Dumper \%hash;

Assumes serial numbers and iLU numbers are unique

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]

stevio · Feb 17, 2009

Kevin

I've run the code and for some reason, it's not:

1)picking up the serial# if it the product is CCS (the serial is 8 digits instead of 7)

2) or showing the 0026

Result of Hash dump

Code:

$VAR1 = {
          '' => {  [COLOR=red]# should be 75010010 here [/color]
                  '0148' => {
                              'Type' => [
                                          'Non',
                                          'Own'
                                        ],
                              'Port' => [
                                          '1A',
                                          '0A'
                                        ]
                            },
                  '' => { [COLOR=red]# should be 0026 here[/color]
                          'Type' => [
                                      'Own'
                                    ],
                          'Port' => [
                                      '0A'
                                    ]
                        }
                },

Everything else works fine

Awesome work! Thanks

Full data set here

Code:

Product       : CCS
SerialNumber  : 75010010
LUs           : 2

iLU  HDevName OSPathID PathID ChaPort Status     Type
0026 hdisk24  00000    000018 1A      Online     Non
              00001    000019 0A      Online     Own
0148 hdisk29  00000    000020 1A      Online     Non
              00001    000021 0A      Online     Own

Product       : SIP
SerialNumber  : 0010111
LUs           : 3

iLU  HDevName OSPathID PathID ChaPort Status     Type
0010 hdisk21  00000    000040 2A      Online     Own
              00001    000041 1A      Online     Own
0020 hdisk26  00000    000017 1A      Online     Own
0021 hdisk22  00000    000039 2A      Online     Own

Product       : SIP
SerialNumber  : 0025024
LUs           : 13

iLU  HDevName OSPathID PathID ChaPort Status     Type
0204 hdisk5   00000    000035 1A      Online     Own
0500 hdisk2   00000    000038 1A      Online     Own
0510 hdisk3   00000    000034 1A      Online     Own
0600 hdisk4   00000    000042 1A      Online     Own
0900 hdisk6   00000    000036 1A      Online     Own
              00001    000037 2A      Online     Own
0901 hdisk7   00000    000032 1A      Online     Own
              00001    000033 2A      Online     Own
0902 hdisk8   00000    000007 1A      Online     Own
              00001    000008 2A      Online     Own
0903 hdisk9   00000    000028 1A      Online     Own
              00001    000029 2A      Online     Own
0904 hdisk10  00000    000011 1A      Online     Own
              00001    000012 2A      Online     Own
0905 hdisk11  00000    000001 1A      Online     Own
              00001    000002 2A      Online     Own
0906 hdisk12  00000    000015 1A      Online     Own
              00001    000016 2A      Online     Own
0907 hdisk13  00000    000013 1A      Online     Own
              00001    000014 2A      Online     Own
0908 hdisk28  00000    000043 1A      Online     Own

Product       : SIP
SerialNumber  : 0080025
LUs           : 8

iLU  HDevName OSPathID PathID ChaPort Status     Type
0000 hdisk25  00000    000000 1A      Online     Own
001B hdisk20  00000    000022 2A      Online     Own
              00001    000023 1A      Online     Own
0030 hdisk15  00000    000024 2A      Online     Own
              00001    000025 1A      Online     Own
0031 hdisk16  00000    000005 2A      Online     Own
              00001    000006 1A      Online     Own
0032 hdisk17  00000    000009 2A      Online     Own
              00001    000010 1A      Online     Own
0033 hdisk18  00000    000026 2A      Online     Own
              00001    000027 1A      Online     Own
0034 hdisk19  00000    000003 2A      Online     Own
              00001    000004 1A      Online     Own
0035 hdisk14  00000    000030 2A      Online     Own
              00001    000031 1A      Online     Own
KAPL01001

ejaggers · Feb 18, 2009

Sure stevio,

1. $Key = "$product,$serial,$iLU"; # found first record not type CCS
2. if ( $hash{$Key} ) { # if found above (step 1).
3. $hash{$Key} .= $_; # add second record not type CCS
4. undef $key; # undef step 1 to look for another one

There's probable a cleaner way to do this, but this was quick

KevinADC · Feb 18, 2009

When I run the data you posted with the code I posted the only problem is one of the iLUs has a non digit in it:

[red]001B[/red] hdisk20

that causes it to be turned into 0 (zero) when converted into an integer. But that is easily fixed by removing zeros instead of treating the iLUs like a number.

Change this line:

$ilu = int($ilu) if ($prod eq 'SIP');

change to:

$ilu =~ s/^0+// if ($prod eq 'SIP');

and that solves that problem. The output I get from the code with the above line changed and your full data:

Code:

$VAR1 = {
          '0025024' => {
                         '905' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '204' => {
                                    'Type' => [
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A'
                                              ]
                                  },
                         '902' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '904' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '600' => {
                                    'Type' => [
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A'
                                              ]
                                  },
                         '908' => {
                                    'Type' => [
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A'
                                              ]
                                  },
                         '906' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '510' => {
                                    'Type' => [
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A'
                                              ]
                                  },
                         '900' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '907' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '500' => {
                                    'Type' => [
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A'
                                              ]
                                  },
                         '901' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  },
                         '903' => {
                                    'Type' => [
                                                'Own',
                                                'Own'
                                              ],
                                    'Port' => [
                                                '1A',
                                                '2A'
                                              ]
                                  }
                       },
          '0010111' => {
                         '21' => {
                                   'Type' => [
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A'
                                             ]
                                 },
                         '10' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '20' => {
                                   'Type' => [
                                               'Own'
                                             ],
                                   'Port' => [
                                               '1A'
                                             ]
                                 }
                       },
          '75010010' => {
                          '0148' => {
                                      'Type' => [
                                                  'Non',
                                                  'Own'
                                                ],
                                      'Port' => [
                                                  '1A',
                                                  '0A'
                                                ]
                                    },
                          '0026' => {
                                      'Type' => [
                                                  'Non',
                                                  'Own'
                                                ],
                                      'Port' => [
                                                  '1A',
                                                  '0A'
                                                ]
                                    }
                        },
          '0080025' => {
                         '35' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '' => {
                                 'Type' => [
                                             'Own'
                                           ],
                                 'Port' => [
                                             '1A'
                                           ]
                               },
                         '33' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '32' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '1B' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '34' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '30' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 },
                         '31' => {
                                   'Type' => [
                                               'Own',
                                               'Own'
                                             ],
                                   'Port' => [
                                               '2A',
                                               '1A'
                                             ]
                                 }
                       }
        };

The very last line of the data file could be a problem:

KAPL01001

if it ever started with a digit or a space and a digit.

I see another problem, this line:

Code:

[red]0000[/red] hdisk25  00000    000000 1A      Online     Own

Note that the iLU is all zeros, so if we remove the leading zeros it is now a null field since its in an SIP section. How do you want to handle that situation?

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]

stevio · Feb 18, 2009

Hi Kevin,

The last line always ends with that format KAPL01001

With this potential problem if ilU being all zeros

Code:

0000 hdisk25  00000    000000 1A      Online     Own

can't I change from this

Code:

if ($line =~ /^\d/) {

to this, as iLU is always 4 characters and there's always a space afterwards...?

Code:

if ($line =~ /^\w{4}\s+/) {

BTW, pls don't apologize for any delays. I am very thankful for the help. Don't know if you get any zzzz.

KevinADC · Feb 18, 2009

Changing the line will not help because you said if the iLU is in an SIP section you want to remove the leading zeros. Removing the zeros is the problem, not finding the lines with the numbers. What do you want to do if its only zeros and falls within an SIP section?

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]

stevio · Feb 18, 2009

Kevin,

I think said in my posts that if iLU in the CCS section has leading zeros, then truncate, not SIP. It will never be all zeros in the CCS section so we are all good

Thanks
Stevio

KevinADC · Feb 18, 2009

1) if product is CCS (serial is 8 digits), truncate leading zeros and print

You sure did, my mistake.

In the code I posted, change this line:

$ilu = int($ilu) if ($prod eq 'SIP');

change to:

$ilu = s/^0+// if ($prod eq 'CCS');

And all should be well.

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]

stevio · Feb 22, 2009

Kevin, trying to do a relatively simple task, print each serial number

Code:

for $serial(keys %hash){
    $value = $hash{$serial}; 
    print "$value\n";
}

__RESULT__

HASH(0x300e9670)
HASH(0x30020d20)
HASH(0x30020c18)
HASH(0x30020b1c)

How do I get the correct values to print a s I get the hex values instead? Why is it doing this?

Eventually want to traverse the individual elements and print them.

Code:

for $serial (keys %hash) {
      for $lus (keys %{$hash{$serial}}){
         for $type (keys %{$hash{$serial}{$type}}) {
              print "Type is $_"; #individual elements
         }
         
      }
  }

KevinADC · Feb 23, 2009

Code:

for my $serial (keys %hash){
    print $serial,"\n"; 
}

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]

Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Regex help please 1

stevio

Vendor

KevinADC

Technical User

stevio

Vendor

ejaggers

Programmer

stevio

Vendor

KevinADC

Technical User

stevio

Vendor

ejaggers

Programmer

KevinADC

Technical User

stevio

Vendor

KevinADC

Technical User

stevio

Vendor

KevinADC

Technical User

stevio

Vendor

KevinADC

Technical User

Similar threads

Part and Inventory Search

Sponsor