Out of Memory for a perl script

[jliang]

Hi,

I am new for perl. I wrote a perl script to replace the variables in a file. But I got error message: out of memory during "large" request for 67112960 bytes, total sbrk() is 5152738
8 bytes at ../autobackend/bin/backend.plx line 30, <READ_FSTLD_FILE> line 1.

I do not understand this error message. Following are the codes:

#!/usr/bin/perl -s

$run_cmd = "|fastload >> ../autobackend/log/" . $OutFile . "_load.log 2>&1";

open(READ_FSTLD_FILE, "$fstldfile") || die "\n Could not open file: $fstldfile for read";

open(TO_FSTLD_FILE, $run_cmd) || die "\n Could not open file fastload file for write";

while ($current_line = <READ_FSTLD_FILE>) {
chop($current_line);
$current_line =~ s/$UNAME/$Uname/g;
$current_line =~ s/$PASSWD/$Passwd/g;
$current_line =~ s/$TARGET_TBL/$Target_tbl/g;
$current_line =~ s/$ERROR_TBL1/$Error_tbl1/g;
$current_line =~ s/$ERROR_TBL2/$Error_tbl2/g;
$current_line =~ s/$FILENAME/$FileName/g;
$current_line =~ s/$LINK_CD/$OutFile/g;
print TO_FSTLD_FILE "$current_line\n";
print "$current_lines\n";
}

close(READ_FSTLD_FILE);

When I run this perl file, pass the arguments like:

../autobackend/bin/backend.plx -s -fileoption="fastload" -fstldfile="../autobackend/tmp/run_test.load" -Uname="xxxx" -passwd="xxxx" -Target_tbl="test" -Error_tbl1="test_err1" -Error_tbl2="test_err2" -FileName="testperl.txt" -OutFile="testxx"

Many thanks in advance.

J

 

[PaulTEG]

$current_line(s)

Looks like a typo, have you declared $/ anywhere in your prog.

It's important in life to always strike a happy medium, so if you see someone with a crystal ball, and a smile on their face ... smack the fecker

 

[MikeLacey]

Paul's comments first, fix $comments/$comment and check that $/ hasn't been set anywhere (it will cause your script to try and read the entire file in one go if it is)

If that doesn't fix problem - have a look at the input file, run_test.load from the look of it; is it a normal text file with a new-line character at the end of each line?

If each line doesn't end with a new-line character you'll have to tell Perl which character each line ends with - by setting $/.

Mike

"Deliver me from the bane of civilised life; teddy bear envy."

Want to get great answers to your Tek-Tips questions? Have a look at faq219-2884

 

[waiterm]

try making use of any array rather than reading every line directly from the file, for example:

open(READ_FSTLD_FILE, "$fstldfile") || die "\n Could not open file: $fstldfile for read";
@current_lines = <READ-FSTLD_FILE>;
close READ_FSTLD_FILE;
foreach $current_line(@current_lines)  {
   ... carry out your code ...
}

It actually looks as though the problem is that the entire file contents are being read into memory for each line you're looping through, so eventually you memory fills up. Array manipulation is a lot more efficient than direct file access.

Rob Waite

http://www.hostpipe.co.uk