sed crashes after special character in command

[zinced]

Binary package hint: sed

wold0126<22>Incoming% lsb_release -rd
Description: Ubuntu karmic (development branch)
Release: 9.10

wold0126<23>Incoming% apt-cache policy sed
sed:
  Installed: 4.2.1-1
  Candidate: 4.2.1-1
  Version table:
 *** 4.2.1-1 0
        500 http://us.archive.ubuntu.com karmic/main Packages
        100 /var/lib/dpkg/status

On a large file of sed commands an accented character causes sed to exit with this message:

cat textfile | sed -f foo.sed

sed: file foo.sed line 4251: unterminated `s' command

The offending line in this file is

s/\<caFé\>/café/g

A file with this as the only line works fine. This effect was not present in 9.04

This may be related to https://bugs.launchpad.net/ubuntu/+source/sed/+bug/447866

[Paolo Bonzini (bonzini)]

What is the exact content (in binary) of the script?

What is $LANG, $LC_ALL and $LC_CTYPE?

[Paolo Bonzini (bonzini)]

(If a single line works while the full script doesn't, this might be a bug in sed indeed).

[zinced (headers)]

Paolo,

Here are the env variables from a system on which I have this issue (Karmic,
64 bit, Intel):

ripper<501>~% echo $LANG
en_US.UTF-8
ripper<502>~% echo $LC_ALL

ripper<503>~% echo $LC_CTYPE

Enclosed is a simplified example of a sed script file (test.sed) that causes
the error.
You will see that is a trite silly example but it is representative of the
error.

ripper<506>~% /bin/sed --version
GNU sed version 4.2.1

echo foofoo | sed -f test.sed will reproduce the error

Thanks,
             Simon.

On Wed, Nov 11, 2009 at 12:33 PM, Paolo Bonzini <email address hidden> wrote:

> (If a single line works while the full script doesn't, this might be a
> bug in sed indeed).
>
> --
> sed crashes after special character in command
> https://bugs.launchpad.net/bugs/452620
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “sed” package in Ubuntu: New
>
> Bug description:
> Binary package hint: sed
>
>
> wold0126<22>Incoming% lsb_release -rd
> Description: Ubuntu karmic (development branch)
> Release: 9.10
>
> wold0126<23>Incoming% apt-cache policy sed
> sed:
> Installed: 4.2.1-1
> Candidate: 4.2.1-1
> Version table:
> *** 4.2.1-1 0
> 500 http://us.archive.ubuntu.com karmic/main Packages
> 100 /var/lib/dpkg/status
>
>
> On a large file of sed commands an accented character causes sed to exit
> with this message:
>
> cat textfile | sed -f foo.sed
>
> sed: file foo.sed line 4251: unterminated `s' command
>
> The offending line in this file is
>
> s/\<caFé\>/café/g
>
> A file with this as the only line works fine. This effect was not present
> in 9.04
>
> This may be related to
> https://bugs.launchpad.net/ubuntu/+source/sed/+bug/447866
>

[Paolo Bonzini (bonzini)]

The error message is ugly, but the bug is that the script is not valid UTF-8, it is encoded as ISO-8859-1. When you create the line in a new script, vi saves it as UTF-8 for you.

Run sed with LANG=en_US.ISO-8859-1 or with LANG=C instead.