Offset error with autocompletion (with color+newline in PS1)

[Daniel Hahler]

Binary package hint: bash

Recipe to reproduce:
1. Set PS1:
$ PS1="\[\033[0;32m\]{ \[\033[0;32m\]\w\[\033[0;32m\] }\n\$\[\033[0m\] "
{ /tmp }
$ _
2. Type "ls ./_ /foo" ("_" is where the cursor should be, just after "./".
3. Press TAB twice.
A list of available files in the current directory should get displayed and the internal cursor position (and the cursor itself) gets moved 11 chars to the right.

This is what the prompt looks like then ("_": cursor):
{ /tmp }
$ ls ./ /foo _

It appears that this bug gets triggered by color/non-visible chars _and_ a newline (\n) in PS1.

This is with 3.2.17(1)-release (3.2-0ubuntu8) (backported from Gutsy).

If someone can confirm this I will send it upstream.

[Björn Lindqvist (bjourne)]

I can confirm this bug using 3.2.13(1)-release (i486-pc-linux-gnu)! Following the above steps reproduces the buggy behavior. Here is another way to reproduce the bug:

1. Set PS1="\033[01;37m[ \[\033[01;34m\]\w\[\033[00m\] \033[01;37m]\n\$\033[00m "
2. Type "foo bar baz"
3. Press ctrl+a
3. Cursor should jump back to the beginning of the line, but instead jumps back to "foo b." This is how it looks like with "_" being the cursor:

foo b_ar baz

[Daniel Hahler (blueyed)]

I've sent this also to the bug-bash mailinglist and Andreas Schwab replied:
-------------
> 1. Set PS1="\033[01;37m[ \[\033[01;34m\]\w\[\033[00m\] \033[01;37m]\n\$\033[00m "
You need to bracket _every_ nonprinting sequence of characters with \[\].
-------------

I'm not sure what he meant and asked him, but no reply yet.

[Daniel Hahler (blueyed)]

I've looked at it again and found that the shortest test code for this is:
1. PS1="\n\[\033[0m\]"
2. "ls ./_ /foo"
3. Press TAB twice, where the "_" is above

I've also tried bracketing the \n itself, although it's a printing character: no difference.

[Daniel Hahler (blueyed)]

Chet Ramey sent me a patch, which fixes this for me.

[Launchpad Janitor (janitor)]

This bug was fixed in the package bash - 3.2-0ubuntu12

---------------
bash (3.2-0ubuntu12) hardy; urgency=low

  * Apply upstream patches 026 - 033. Fixes for:
    - Keep the Apple linker from attempting to link bash against Apple's
      readline library "replacement" rather than the one shipped with bash.
    - When updating the display after displaying, for instance, a list
      of possible completions, readline will place the cursor at the wrong
      position if the prompt contains invisible characters and a newline.
      LP: #119938.
    - Under some circumstances, readline will incorrectly display a prompt
      string containing invisible characters after the final newline.
    - When the bash arithmetic expression evaluator has temporarily turned off
      evalation, such as when parsing a pre- or post-decrement or -increment
      operator, and an error occurs, evaluation is not re-enabled.
    - If redirections attached to a compound command fail, bash does not set
      the command's exit status correctly. This only happens when the command
      is the first in a sequential list.
    - In certain cases when outputting characters at the end of the line, e.g.,
      when displaying the prompt string, readline positions the cursor
      incorrectly if the prompt string contains invisible characters and the
      text being drawn begins before the last invisible character in the line.
    - There is an off-by-one error in the code that buffers characters
      received very quickly in succession, causing characters to be dropped.
    - References made within a function to an uninitialized local array
      variable using the [*] subscript in a double-quoted string can result
      in spurious ASCII 127 characters in the expanded value.
  * Remove bash-completion from the source.
  * Remove the conflict with bash-completion, recommend bash-completion.
  * debian/skel.bashrc: Check for dircolors before using it. LP: #121992.
  * Include correct bash.pot file. LP: #104261.

 -- Matthias Klose <email address hidden> Fri, 08 Feb 2008 19:26:17 +0100

[Arto Karppinen (arto-karppinen-mail)]

Im reopening this because with bash 3.2-0ubuntu18 im having this same problem.

Recipe written by Björn Lindqvist wrote on 2007-06-30 still breaks bash color prompt:

1. Set PS1="\033[01;37m[ \[\033[01;34m\]\w\[\033[00m\] \033[01;37m]\n\$\033[00m "
2. Type "foo bar baz"
3. Press ctrl+a
3. Cursor should jump back to the beginning of the line, but instead jumps back to "foo b." This is how it looks like with "_" being the cursor:

foo b_ar baz

[Anders Kaseorg (andersk)]

Björn’s PS1 is broken as per comment #2: “You need to bracket _every_ nonprinting sequence of characters with \[\]”. In this case,

PS1="\[\033[01;37m\][ \[\033[01;34m\]\w\[\033[00m\] \[\033[01;37m\]]\n\$\[\033[00m\] "

I cannot reproduce with this fixed PS1. Do you have another example of this broken behavior?

[Anders Kaseorg (andersk)]

Ah, I do have an example. Use the default colorized prompt from /etc/skel/.bashrc, and cd into a directory with a very long pathname that causes the prompt to linewrap:

PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
ln -s . /tmp/long-path
cd /tmp/long-path/long-path/long-path/long-path/long-path/long-path/long-path/long-path

Then, if you type a very long command that results another linewrap, and hit Ctrl+A, the cursor will be moved to the sixth character of the command instead of the first, and future editing operations will affect the wrong characters.

anders@balanced-tree:/tmp/long-path/long-path/long-path/long-path/long-path/long
-path/long-path/long-path$ long-_command long-command long-command long-command l
ong-command

Reproduced on both Hardy’s bash 3.2-0ubuntu18 and Intrepid’s bash 3.2-4ubuntu1.

[Arto Karppinen (arto-karppinen-mail)]

Ah, yes. The default colorized prompt is also the way i came across this bug.

And then found this bugreport, and then tested the exinsting recipe instead of writing my own. Sorry about that :(

I also noticed that if you take the colorization away from the path in the prompt then the problem goes away:

PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\w\$ '

But if i remove the colorization from the user@host part the problem remains:

PS1='${debian_chroot:+($debian_chroot)}\u@\h:\[\033[01;34m\]\w\[\033[00m\]\$ '

[Anders Kaseorg (andersk)]

This all seems to be fixed in current bash.