How do read STDOUT and STDERR together

[wardy66]

Hello

I have a (hopefully) small problem reading output from a command that throws off both STDOUT and STDERR. When run on the terminal, these two files are interspersed correctly.

The output takes the form of:
STDOUT blah blah blah
STDOUT processing file2
STDERR processing failed
STDOUT processing file3
STDOUT processing file4
etc etc

The problem is that when read, the output seems to be buffered, ie, STDOUT comes out together and STDERR comes out together - not interspesed. So, the error is meaningless unless I can see what the STDOUT line above is.

Is there a way to emulate an interactive terminal when running this command? That way, I'm hoping the output will be able to be processed in the same order as it appears on a terminal.

I've tried using 2>&1 inside `` and system() but to no avail.

Does anyone know how to get this to work?

Thanks
~ Michael

 

[mikevh]

If you redirect the output of the command to a file, does it come out interspersed, as it does on a terminal?

 

[wardy66]

Nope, it doesn't work either. I think what's happening is that the command we're running is buffering the output but it doesn't respect Perl's $| setting.

Here's a sample Perl prog I wrote to help with the testing:

[tt]
$ cat output.pl
$| = @ARGV;

print "I've set \$| to $|\n\n";


for ( $i = 1; $i < 12; $i++ ) {
$fh = int(rand(2)) ? STDOUT : STDERR;
print $fh "$i: Here is some output to file $fh\n";
}

$ perl output.pl > a.txt 2>&1
$ perl output.pl b > b.txt 2>&1

$ head -30 [ab].txt
==> a.txt <==
3: Here is some output to file STDERR
10: Here is some output to file STDERR
11: Here is some output to file STDERR
I've set $| to 0

1: Here is some output to file STDOUT
2: Here is some output to file STDOUT
4: Here is some output to file STDOUT
5: Here is some output to file STDOUT
6: Here is some output to file STDOUT
7: Here is some output to file STDOUT
8: Here is some output to file STDOUT
9: Here is some output to file STDOUT

==> b.txt <==
I've set $| to 1

1: Here is some output to file STDOUT
2: Here is some output to file STDERR
3: Here is some output to file STDOUT
4: Here is some output to file STDERR
5: Here is some output to file STDOUT
6: Here is some output to file STDERR
7: Here is some output to file STDOUT
8: Here is some output to file STDOUT
9: Here is some output to file STDOUT
10: Here is some output to file STDOUT
11: Here is some output to file STDERR
[/tt]

You can see that buffering changes the order we write to the file dramatically. So, I think we need the command we're running to not buffer its output.

So, I'm guessing we need to trick it into thinking we're a terminal.

Any idea how?

Or does someone have another suggestion?

Thanks
~ Michael

 

[mikevh]

I may be wrong here (I don't have access to a Unix box currently), but to redirect both STDOUT and STDERR to file a.txt, shouldn't that be
$ perl output.pl 2>&1 > a.txt
rather than
$ perl output.pl > a.txt 2>&1

Also, I don't get
$ perl output.pl b > b.txt 2>&1
at all. What is the argument "b" doing here?

My Unix is a bit rusty, so if I'm way off here, let me know.

 

[wardy66]

Thanks for the reply, mikevh

As for the "b" option, it was just my lazy way of testing the effect of $| in Perl, ie, the "$| = @ARGV;" bit. Supplying any option to the script sets $| and hence turns off buffering.

I did just just to see if that was indeed the problem.


Also, if you do "perl output.pl 2>&1 > a.txt" then you will send STDERR to the termincal and STDOUT to the file. I want to get both together - but as you can see from above I want them in the order issued which is proving to be the problem