Regex Puzzle

[audiopro]

I have a bit of code which does a substitution within a loop but the code 'locks up'. If I remove the substitution line, the code works fine. To check the syntax of the sub statement I wrote this....

my $TESTER="In my Student days I was a STUDENT with many student friends who knew other StUdEnTs";
$TESTER=~ s/student/wasane/ig;
print "$TESTER";

All versions of the word student are replaced as expected.
When I use the same substit. on a database field (text length = 7775) the script locks up and I get no error message.
Is there a limit to the string length which can be handled?
Would the fact there are line feeds, in the text, affect it?


Keith

http://www.studiosoft.co.uk

 

[audiopro]

I have tried a number of variations including

$TESTER=~ s/student/wasane/igm;

but the script still locks up

Keith

http://www.studiosoft.co.uk

 

[MillerH]

There is nothing special in your regular expression that could possibly lead to it falling into an infinite loop. This means that there must be something in either the data itself in the database, or in the way that you are accessing it.

I suggest that you try this to debug the problem. Put the data in a new variable and try the regex on it. If that doesn't work, then take a 1 character substr of it and try the regex on it. If that DOES work, then try a 2 character subsr. Then a 3. All the ways until you are either working on the whole data, or the regex fails at a specific character point. If it fails, you can look to see what occurs at that character that might be lead to this type of error. And if it succeeds all the way to the end, then there must be something bad in the way that you're accessing the data.

Obviously do the looping of the substr programmatically. And don't worry about the regex modifiers. m + s as those aren't the nature of your problem.

Good Luck.

 

[KevinADC]

also.... check the server error logs.

------------------------------------------
- Kevin, perl coder unexceptional!

 

[audiopro]

I still can't get to the bottom of the problem.
I have written the script in a different way in order to get it working but I have encountered this lock up in a number of instances. I will do systematic regex to find the cause but as always the bill payer wants a working script not a theory as to why it doesn't work.
I was just checking with you guys if there was a limit to the length of string a regex will work on. I will post the results here just in case anyone else encounters this problem.


Keith

http://www.studiosoft.co.uk

 

[MillerH]

There is no limit to the length of a string a regex will work on, No.

 

[audiopro]

Thanks for that.
Kevin - The error log does not report anything.
After I run the script, it just locks up the resulting HTML page looks like it is trying to load something but nothing happens. I can get similar behaviour if I fail to terminate a string ie.

print "This string is started but not finished;

Not something I've ever done for real of course! LOL
That too does not give me an error, just a lock up and leaves the Perl process open. I don't understand why the process does not fail and die in some way.
Is there a setting somewhere which will close the open process and write to the log file?

Keith

http://www.studiosoft.co.uk

 

[MillerH]

yes, overall this sounds like a bug in your environment. Whenever you have syntax errors of the nature of unterminated strings, perl should fail in a complete way and report the error.

- M

 

[audiopro]

It fails no problem on non declared vars in strict and missing brackets etc. but has always locked up on unterminated strings.
Should I take it outside and shoot it?

Keith

http://www.studiosoft.co.uk

 

[MillerH]

Should I take it outside and shoot it?

I'm not sure that violence is the answer just yet, as much as I love random gratuitous canon fire.

I'm simply suggesting that it would be in your own benefit to create the side project of decided how to get perl to more properly handle compiling errors like unterminated strings. Unfortunately, I've never heard or such a circumstance when perl would choose to just lock up, so I can't suggest what might be wrong. In all of my environments perl will return an error about an unterminated string or missing semicolon or such:

Can't find [b]string terminator[/b] "'" anywhere before EOF at space.pl line 3.

or

Bareword found where operator expected at space.pl line 11, near "print "hello"
  [b](Might be a runaway multi-line "" string starting on line 7)[/b]
        (Do you need to predeclare print?)
syntax error at space.pl line 11, near "print "hello world"
Can't find string terminator '"' anywhere before EOF at space.pl line 11.

Obviously, none of this doesn't directly resolve your original issue though. There is nothing that I can add to that based off of the information that you've given us so far.

Good Luck.

 

[KevinADC]

can we see the real line (or lines) of code that is causing the problem?


------------------------------------------
- Kevin, perl coder unexceptional!

 

[audiopro]

I've had the second error but not the first.
Where are errors handled, within Perl or a seperate handler script?

Keith

http://www.studiosoft.co.uk

 

[MillerH]

Where are errors handled, within Perl or a seperate handler script?

Assuming that you're talking about web development, I use mod_perl.

http://perl.apache.org/

I prefer this environment because all errors are reported at compile time instead of runtime like a lot of other environments. In other words, I can't start apache while there are errors in any modules that I use.

It sounds like you've using scripts though, which are not compiled in advance and instead only report errors at runtime. I'm not sure the best method to get perl to return errors earlier in such an instance. I suppose one method would be to create a test script that mimics a web environment and then compiles each of your scripts. Regardless though, I have no explanation as for why perl would choose to "lock up" instead of reporting that there is a syntax error.

I have limited experience with other development environments, so can't really suggest anything. I was simply wanting to point out my concern when you said that perl just locks up, and that I would suggest that you look into ways of fixing that situation.

Shrug.

Also, if you follow Kevin's advice and show us some real code, we might at least be able to help you debug the regex issue.

 

[KevinADC]

I've seen what looks like perl locking up when there are tons and tons of warnings being generated, especially in some sort of loop or recursive process. The script is still running but simply can't get past issuing so many warnings it looks like it has just stopped. Especially with "use of uninitialized variable used in blah blah" type warnings.

------------------------------------------
- Kevin, perl coder unexceptional!

 

[MillerH]

I've seen what looks like perl locking up when there are tons and tons of warnings being generated, especially in some sort of loop or recursive process. The script is still running but simply can't get past issuing so many warnings it looks like it has just stopped. Especially with "use of uninitialized variable used in blah blah" type warnings.

I've never run into that. I suppose I follow the mantra of never make a programming mistake.

LoL, just kidding.

 

[audiopro]

This is one example of a failing script from a previous version. I am sure that you guys will say this could be done simpler but it was an evolved script which was later condensed. The point being, it should still work.

The web pages within this site use database stored templates to create whole pages, this sub routine displays all the templates, with test text inserted.

I have put some explan. in the script but basically the problem concerns the substitutions.
If I take them out the script runs fine. I thought the problem may be due to inserting HTML codes, I removed those commands but it made no difference.

# ------------------------------------------------------------------------------------------------
sub PageExam{
# ------------------------------------------------------------------------------------------------
my $elim;
my $stysql;
my $stysth;
my $stylist;
my $tempcount;
# Turn off headers and footers for page layout
$PageExample='pagexamp';

# Get data from table
	$PRIFMESS=&create_prif('oindex.htm','<!-- COMMENCE HTML -->','<!-- START CONTENT BODY -->',"Menu Option",1);


	my $xsql="SELECT * FROM $TABLE_NAME[7]";
	my $xsth=$dbh->prepare($xsql);
	$xsth->execute();
	my $xrv=$xsth->rows;
	my @examples;

# $PAGE_TEXT- Code inserted into place holders

	my $PAGE_TEXT="We can enhance the text by using CSS Text control codes<br>";


# Removed in case there is some issue with inserting HTML codes

#	$PAGE_TEXT.="<h1>h1 - text</h1><br>";
#	$PAGE_TEXT.="<h2>h2 - text</h2><br>";
#	$PAGE_TEXT.="<h3>h3 - text</h3><br>";
#	$PAGE_TEXT.="<h4>h4 - text</h4><br>";
#	$PAGE_TEXT.="<h5>h5 - text</h5><br>";
#	$PAGE_TEXT.="<p>p - text</p><br>";


# Image code to insert into image placeholders

	my $PAGE_IMAGE="<img border='0' src='bitims/hol2.jpg'>";
	while(@examples=$xsth->fetchrow_array()){
		print "<div class='exhead'>$examples[1]</div>";
		for($elim=2; $elim<@examples; ++$elim){
			chomp $examples[$elim];
			if(substr($TEESTEM_TEXT[$elim],0,7) eq 'SEGTYPE'){
				if($examples[$elim] ne '-------'){
					$stysql="SELECT STYLE FROM $TABLE_NAME[6] WHERE NAME='$examples[$elim]'";
					$stysth=$dbh->prepare($stysql);
					$stysth->execute();
					$stylist=$stysth->fetchrow_array();

# Remove substitutions and script works ok
# Leave them in and it just locks up
# Leave any single one in and it locks up
# Same result if the match statements are removed and just the substitutions left


# Replace Text
					if($stylist=~ m/ZZTEXT1/){$stylist=~ s/ZZTEXT1/$PAGE_TEXT/gm}
					if($stylist=~ m/ZZTEXT2/){$stylist=~ s/ZZTEXT2/$PAGE_TEXT/gm}
					if($stylist=~ m/ZZTEXT3/){$stylist=~ s/ZZTEXT3/$PAGE_TEXT/gm}
					if($stylist=~ m/ZZTEXT4/){$stylist=~ s/ZZTEXT4/$PAGE_TEXT/gm}
					if($stylist=~ m/ZZTEXT5/){$stylist=~ s/ZZTEXT5/$PAGE_TEXT/gm}

#Replace Images
#					if($stylist=~ m/ZZIMAGE1/){$stylist=~ s/ZZIMAGE1/$PAGE_TEXT/g}
#					if($stylist=~ m/ZZIMAGE2/){$stylist=~ s/ZZIMAGE2/$PAGE_TEXT/g}
#					if($stylist=~ m/ZZIMAGE3/){$stylist=~ s/ZZIMAGE3/$PAGE_TEXT/g}
#					if($stylist=~ m/ZZIMAGE4/){$stylist=~ s/ZZIMAGE4/$PAGE_TEXT/g}
#					if($stylist=~ m/ZZIMAGE5/){$stylist=~ s/ZZIMAGE5/$PAGE_TEXT/g}


# show explanations, hidden in template

					$stylist=~ s/<!--/ /g;
					$stylist=~ s/-->/ /g;
					print "$stylist";
				}
			}
		}

	}

# Turn on headers and footers for page layout
	$PageExample='';
	$PRIFMESS=&create_prif('oindex.htm','<!-- END CONTENT BODY -->','<!-- FINISH HTML -->',"Menu Option",2);
	exit;
}

If only if it was possible

I suppose I follow the mantra of never make a programming mistake.

Keith

http://www.studiosoft.co.uk

 

[KevinADC]

this looks wrong:

$stylist=$stysth->fetchrow_array();

probably should be:

@stylist=$stysth->fetchrow_array();

<quoted from DBI module:>

fetchrow_array

@ary = $sth->fetchrow_array;

An alternative to fetchrow_arrayref. Fetches the next row of data and returns it as a list containing the field values. Null fields are returned as undef values in the list.

If there are no more rows or if an error occurs, then fetchrow_array returns an empty list. You should check $sth->err afterwards (or use the RaiseError attribute) to discover if the empty list returned was due to an error.

If called in a scalar context for a statement handle that has more than one column, it is undefined whether the driver will return the value of the first column or the last. So don't do that. Also, in a scalar context, an undef is returned if there are no more rows or if an error occurred. That undef can't be distinguished from an undef returned because the first field value was NULL. For these reasons you should exercise some caution if you use fetchrow_array in a scalar context.


------------------------------------------
- Kevin, perl coder unexceptional!

 

[audiopro]

Cheers Kevin
I have used the scalar method numerous times to get single column data without problems but if that way is not reccomended I'll return an array in the normal way.
I have changed it to fetch an array and substitute $stylist[0] but the lock up is still happening. I will look into some more when I have a bit of time.

Keith

http://www.studiosoft.co.uk