Flushing the output of an interrupted command

[Clouseau2]

Hi - first post here. I'm trying to run commands in perl and have them not take longer than a certain amount of time, so things can't hang forever. I also want to be able to read all the output, even if I interrupt the command. I've written a test script and it's not working. I can read all the stderr output, but not stdout.

Here is the test command:

#!/bin/perl

print STDOUT "stdout\n";
print STDERR "stderr\n";
print STDOUT "stdout\n";
print STDERR "stderr\n";
sleep $ARGV[0];

And here is how I invoke it:

#!/usr/bin/perl

my $timeout = $ARGV[0];
my @output;

eval {

    local $SIG{ALRM} = sub {
        close (COMMAND); die;
    };

    alarm $timeout;
    if (open (COMMAND, "(perl /tmp/sleep.perl 3 </dev/null | sed 's/^/STDOUT:/') 2>&1 |")) {
#    if (open (COMMAND, "perl /tmp/sleep.perl 2 </dev/null 2>&1 |")) {

        my $old_fh = select (COMMAND);
        $| = 1;
        select ($old_fh);

        while (<COMMAND>) {
            print "input: $_";
            push (@output, $_);
        }
    }
    close (COMMAND);
    alarm 0;
};

if ($@) {
    print "problem\n";
} else {
    print "no problem\n";
}

foreach $index (0 .. $#output) {
    print "$index) $output[$index]";
}

Note: the sed step is so I can separate STDOUT and STDERR output, but the result is the same either way. The < /dev/null is so that any user input is skipped. Changing these things has no effect.

I tried adding IO::Select stuff to read the rest of the output when it is interrupted, but it didn't make any difference. If I run it like this, I get all the output from the first script:

% perl test.perl 5
input: stderr
input: stderr
(note it pauses here, so the stdout is not being flushed out until the script ends)
input: STDOUT:stdout
input: STDOUT:stdout
no problem
0) stderr
1) stderr
2) STDOUT:stdout
3) STDOUT:stdout


But if I have it be interrupted, I don't get the stdout:

% perl test.perl 1
input: stderr
input: stderr
Broken Pipe
problem
0) stderr
1) stderr


How can I get all the output every time?

As you can see, the first script outputs everything, although the stdout isn't output until the executing script ends.

 

[travs69]

I think this is possibly your problem
(note it pauses here, so the stdout is not being flushed out until the script ends)

You don't have any pauses built in there and you have $|=1 set so I don't know why it isn't printing out up front..

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;

 

[Annihilannic]

I think you need to set $|=1 inside sleep.perl, since it only affects output buffering.

Also you'll need to remove the sed from the pipeline because it may (does in my case) reintroduce output buffering to the pipeline.

Annihilannic.

 

[Clouseau2]

But I can't do that. The sleep.perl script is just a test script, I'm actually going to be running various programs (not perl scripts), I want to make sure I capture as much output from these programs as possible. Why is stdout being buffered and not stderr?

I guess my question is, is there any way for me to force a process to flush the output buffers before I close it?

 

[Annihilannic]

I guess standard output is typically buffered for efficiency, whereas standard error is not for expediency (it's assumed you want to know about errors ASAP, and also that not much output will be sent by that route).

I have had some success just setting $|=1 for STDOUT as you did, mostly in scripts that read the output of tail -f <somelogfile> from STDIN, but I can't identify a reliable pattern of which programmes work with this and which don't. Nor do I understand why modifying STDOUT buffering changes the behaviour of reading STDIN.

I don't believe you'll find any standard way to cause other processes to flush buffers though.


Annihilannic.