Perl Resources: Advice

[Trancemission]

I am currently working on a script that will monitor a logfile continusly. This log file grows currently at a rate of about 3 lines per second. I am using the open() function to open and loop thorug the logfile.

At the moment the script runs fine, the logfile is rotated every day and the maximum size I have experianced is 25MB.

I am quite new to Perl programming and was looking for advice on how large my logfile can be and my perl script will still run fine. I will be runing this on a dedicated Sparc5 box, with 1 GB RAM and 45+GB HDD.

The project rate of messages of this logfile will grow to 1300 messages per second, bring the logfile to towards 6GB?!?

Any advice/readmes/whitepapers would be aprreciated.

Many Thanks






Trancemission
=============
If it's logical, it'll work!

 

[MikeLacey]

Hi there,

There aren't any Perl issues, that I'm aware of, regarding reading large files using Perl.

When processing log files like this I use the technique of processing until I reach the end of file, sleep for a second and then call seek(TEST,0,1); (your filehandle instead of TEST, of course) which has the effect of finding any data that's been added since you reached eof() and went to sleep.

Mike

Want to get great answers to your Tek-Tips questions? Have a look at faq219-2884

It's like this; even samurai have teddy bears, and even teddy bears get drunk.

 

[Trancemission]

Many Thanks for that, I am using the same method.

Cheers

Trancemission
=============
If it's logical, it'll work!

 

[missbarbell]

Try the following:

open LOG, "tail -f $log|" or die "Couldn't open pipe: $!\n";

Saves you having to worry about memory problems, and continually opening/closing the log file.

Barbie
Leader of Birmingham Perl Mongers

http://birmingham.pm.org