Zero-width Character Set 1

[Kirsle]

In bash you can colorize your bash prompt using ANSI colors using a method like this:

PS1="[red]\e[1;31m[/red][\u@\h \W]\$[red]\e[0m[/red] "

This method of getting a colored prompt however messes up the word wrapping of the terminal. Since the entire sequence of "\e[1;31m" actually takes up zero characters when displayed in the terminal (all it does is cause everything that follows to be displayed as bright red), the terminal emulator can't calculate when to word-wrap and it causes symptoms such as:

* The line tries to wrap too soon (before you reach the end of the line)
* When it wraps it actually doesn't break to the next line, but goes back to the start of the current line, so you type over top of what's already been typed
* When it's doing this, if you backspace or correct your command, each correction you make causes the whole entire line to be copied to the next line with the correction made.

The way around this for bash is to use the \[\] format for the PS1 prompt, and use the tput command inside them:

PS1="[red]\[$(tput bold; tput setaf 1)\][/red][\u@\h \W]\$[red]\[$(tput sgr0)\][/red] "

The \[...\] format causes the result of everything inside those brackets to result in a zero-width output, so that the terminal emulator doesn't count the output when it calculates when to do word-wrapping at.

How this relates to Perl

If you make a Perl console application that gives the user a prompt to run commands (a MySQL client for instance), and the prompt is to be colorized, you'd probably use Term::ANSIColor like this:

print Term::ANSIColor::colored("mysql> ", 'bold green');

But this would cause \e[1;32m and \e[0m to be added to the text "mysql> ", which messes with the word-wrapping in the terminal emulator and causes the same problem as in bash.

Is there a Perl equivalent of \[\] so that ANSI color codes can be used without messing with the word wrapping?

Cuvou.com | My personal homepage

perl -e '$|=$i=1;print" oo\n<|>\n_|_";x:sleep$|;print"\b",$i++%2?"/":"_";goto x;'

 

[Annihilannic]

It is not the terminal that handles the wrapping, but bash itself; and it's bash that uses those escape codes to identify which parts of the prompt to count or ignore when calculating the screen width.

So in your perl example it is the routine that performs the getline from the terminal that would need to calculate or be told the available width for the prompt. Most "getline"s are very dumb though and the entered text will just wrap at the edge of the screen and not return to the first line when backspace is used.

So it depends on what method you are using to read the user input?

Annihilannic.

 

[Kirsle]

Just using a simple <STDIN>.

Terminal emulators that have this problem are: gnome-terminal, Konsole, Terminal (xfce), Putty, and also the default text mode terminal in Linux. It might be bash's fault, but the same thing happens when you read in Perl using <stdin> if the line has ANSI codes on it: line breaking happens too soon and you begin typing at the beginning of the same line, instead of the next line.

It's like

Terminal Size:
+---------------------------------+

($> has ANSI codes around it, then you type...)
[b][blue]$>[/blue] The quick brown fox ju[/b]
(then it wraps too soon, not even at the end of the line)

(and then when it wraps it begins looking like this)
[b]mps over teh lrown fox ju[/b]

(made a typo so ya backspace, then yer terminal looks like this)
[b]mps over teh lrown fox ju
mps over teh  rown fox ju [u](hit backspace)[/u]
mps over teh  rown fox ju [u](hit backspace)[/u]
mps over te   rown fox ju [u](hit backspace)[/u]
mps over t    rown fox ju [u](hit backspace)[/u]
mps over the lazy dog x ju [u](finished the sentence)[/u][/b]

So it wraps the line too soon (basically the length of all the hidden ANSI codes away from the edge of the terminal, so 8 ANSI code chars = it wraps 8 characters too soon), and when it wraps you begin typing over the very same line you were on before it wrapped, but then when you backspace, the entire line is then printed again on a new line except with the change you just made, so if you backspace 5 times, the line is copied 5 times with each copy being the new version of the input.

This makes it impossible to go back and correct mistakes because of all the junk on the screen.

So it's pretty much impossible to type something long in a line that has ANSI color codes in it, because of this word wrapping issue.

Bash has the \[\] that you can wrap ANSI colors in so it ignores them when it calculates the width; but since Perl has the same problem Bash does, it would mean it's a problem in the terminal emulator? But since all the major terminal emulators act the same way... it's the terminal that decides when to word-wrap.

Cuvou.com | My personal homepage

perl -e '$|=$i=1;print" oo\n<|>\n_|_";x:sleep$|;print"\b",$i++%2?"/":"_";goto x;'

 

[Kirsle]

Here's a screenshot:

Cuvou.com | My personal homepage

perl -e '$|=$i=1;print" oo\n<|>\n_|_";x:sleep$|;print"\b",$i++%2?"/":"_";goto x;'

 

[Annihilannic]

I concur with everything you say above about the way the bash prompt behaves. But when I try this with perl I can't reproduce the same behaviour... it just behaves like a dumb terminal, with the text wrapping when it reaches the edge of the screen, and being unable to go back and edit the previous line (at least, not visually) once this has happened.

Here is my attempt to reproduce it using gnome-terminal:

Annihilannic.

 

[Kirsle]

Hi,

It looks like the program I was talking about uses Term::Readline to read inputs and not <STDIN>. I thought it was using <STDIN> because the code had this subroutine in it:

sub readline
{
   return <STDIN> || exit;
}

But looking at the code again there are several areas where Term::Readline is used instead:

$line = (
   $indata
   ? shift(@data)
   : $LCL->{TERM}->readline($LCL->psEval($LCL->{CFG}->{PS1}))
);

...

sub psEval {
   my ($LCL, $text) = @_;
   $text = Term::ANSIColor::colored($text, 'bold green');

   return $text;
}

and the PS1 var here is "sql++> ".

So a simpler example of this bug is:

#!/usr/bin/perl -w

use Term::ReadLine;
my $term = new Term::ReadLine 'sql++';

my $prompt = "\e[1;32msql++>\e[0m ";
my $line = $term->readline($prompt);
chomp $line;
print "You said: $line\n";

Looking at Term::Readline's manpage, it seems like it tries various attempts at using a platform-dependent readline() method, sorta like the Sys::Hostname module does to get the hostname in a system-dependent way. So it probably comes down to a Linux readline call being invoked to read a line of input, which isn't aware of the ANSI codes already used on the current line and has the same problem bash does.

Cuvou.com | My personal homepage

perl -e '$|=$i=1;print" oo\n<|>\n_|_";x:sleep$|;print"\b",$i++%2?"/":"_";goto x;'

 

[Annihilannic]

Bingo!

The rl_expand_prompt function in the GNU Readline Library mentions the special markers RL_PROMPT_START_IGNORE and RL_PROMPT_END_IGNORE (declared in `readline.h').

#define RL_PROMPT_START_IGNORE '\001'
#define RL_PROMPT_END_IGNORE '\002'

So this seems to do the trick nicely.

my $prompt = "\001\e[1;32m\002sql++>\001\e[0m\002 ";

At least that's a solution for Linux, not very portable perhaps...

Annihilannic.

 

[Kirsle]

Nice! Thanks. It should be trivial then to check which backend Term::Readline is using and use the \001,\002 only for those cases.

Have a star.

Cuvou.com | My personal homepage

perl -e '$|=$i=1;print" oo\n<|>\n_|_";x:sleep$|;print"\b",$i++%2?"/":"_";goto x;'