Wrong escape character processing in dash

[Martti Kuparinen]

Binary package hint: dash

It seems dash does not handle escape characters correctly. Below is how I can trigger (what I think is) an error on Ubuntu 8.04.1.

# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=8.04
DISTRIB_CODENAME=hardy
DISTRIB_DESCRIPTION="Ubuntu 8.04.1"
# ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2008-03-19 13:13 /bin/sh -> dash
# dpkg -l dash
ii dash 0.5.4-8ubuntu1 POSIX-compliant shell

# cat /tmp/runme
echo "printf(\"Hello\\n\");"

# /bin/bash /tmp/runme
printf("Hello\n");

## THE PROBLEM IS HERE
# /bin/sh /tmp/runme
printf("Hello
");

I also tested this on our server running NetBSD/amd64 and I have no problems with /bin/sh there.

# uname -srm
NetBSD 4.0_STABLE amd64
# ls -l /bin/sh
-r-xr-xr-x 1 root wheel 167620 Aug 13 08:40 /bin/sh

# /usr/pkg/bin/bash /tmp/runme
printf("Hello\n");

# /bin/sh /tmp/runme
printf("Hello\n");

[Ralfixx (ralfixx)]

This also breaks the following script:

  target='\1.o'
  echo foo.c | sed -e 's/\([a-z]*\).c/'$target'/'

Ubuntu /bin/dash:
$ target='\1.o'
$ set -x
$ echo foo.c | sed -e 's/\([a-z]*\).c/'$target'/'
+ echo foo.c
+ sed -e s/\([a-z]*\).c/1.o/
1.o

Ubuntu /bin/bash:
$ target='\1.o'
$ set -x
$ echo foo.c | sed -e 's/\([a-z]*\).c/'$target'/'
+ sed -e 's/\([a-z]*\).c/\1.o/'
+ echo foo.c
foo.o

HP-UX /bin/sh
$ target='\1.o'
$ set -x
$ echo foo.c | sed -e 's/\([a-z]*\).c/'$target'/'
+ echo foo.c
+ sed -e s/\([a-z]*\).c/\1.o/
foo.o

As you can see, sed is called without the backslash in the replacement command when dash is used:
  sed ... /1.o/
instead of the expected
  sed ... /\1.o/

[Mike Sharov (msharov)]

It's actually worse than that. For example, try echoing \1 (slash and one), which shows up often when making sed scripts.
Here's what happens in dash-0.5.5.1:
$ echo '\1'
(outputs 0x01 0x0A, which is only visible with od)
Here it's incorrectly treated as an octal sequence, even though it does not begin with 0. \01 should be required for the above output, and no such processing should happen in single quotes anyway.
$ echo '\\1'
\1
Here It processes the escape sequence again in single quotes. bash would output \\1
$ echo '\\\1'
\
Both backslash and \1 are interpreted.
$ echo "\1"

Outputs 0x01 0x0A again.
$ echo "\\1"

Surprizingly outputs 0x01 0x0A. Where did that come from?

Overall, there simply is no way to output \1 into a sed script that will work in both bash and dash.

[Jilles Tjoelker (jilles)]

You can avoid these problems by using the printf utility instead of echo.

[Martti Kuparinen (mk1970)]

Same bug exists in 10.04...

# ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2010-04-27 13:20 /bin/sh -> dash

# dpkg -l dash
ii dash 0.5.5.1-3ubunt POSIX-compliant shell