grep switches into binary mode while processing a text file under the C locale
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
grep (Debian) |
Fix Released
|
Unknown
|
|||
grep (Ubuntu) |
Fix Released
|
High
|
Martin Pitt | ||
Xenial |
Fix Released
|
High
|
Martin Pitt | ||
Yakkety |
Fix Released
|
High
|
Martin Pitt |
Bug Description
I noticed this staring to happen in Xenial about two days ago. When running sbuild (or now the buildd, too), the build breaks when trying to compile a generated file. I traced the problem down to grep suddenly acting weird. When not having any language set (or a non-UTF8 mode) it will start printing some lines of a source file and then suddenly end that by printing "Binary file ... matches".
With the attached file, the difference can be observed (running Xenial):
LANG=C grep -v xxx grant_table.h
and
LANG=C.UTF-8 grep -v xxx grant_table.h
SRU INFORMATION
===============
Upstream fixes:
- http://
- http://
Test case:
Call grep on a file or a string with non-ASCII characters in the C locale:
$ echo 'héll☺ ≥x' | LC_ALL=C grep .
In xenial this just shows "Binary file (standard input) matches", with the fix it should show the actual input string (with some garbled output of course as the UTF-8 chars cannot be displayed in C)
Regression potential: grep is being used in tons of places; during xenial we had to fix/put a "use grep -a" workaround into a lot of packages to fix the fallout from grep 2.23 which introduced this. That said, as a result of "Binary file matches" does not give any more information than the actual string match, and scripts which get along with this answer most likely just check the exit code anyway (which does not change), the risk is bearable.
We will soon do a test rebuild in yakkety with gcc-6 and grep 2.25, and will sift through the results to identify new FTBFS that are due to grep 2.25. This SRU should not be released until this happens.
tags: | added: wily |
tags: | added: rls-x-incoming |
Changed in grep (Ubuntu): | |
assignee: | nobody → Martin Pitt (pitti) |
description: | updated |
description: | updated |
Changed in grep (Debian): | |
status: | Unknown → Fix Released |
Changed in grep (Ubuntu Xenial): | |
importance: | Undecided → High |
tags: | added: regression-release |
summary: |
- grep switches into binary mode while processing a text file + grep switches into binary mode while processing a text file under the C + locale |
I was unable to recreate the problem given the test case provided and the following version of grep:
$ apt-cache policy grep mirrors. cat.pdx. edu/ubuntu xenial/main amd64 Packages dpkg/status
grep:
Installed: 2.23-1
Candidate: 2.23-1
Version table:
*** 2.23-1 500
500 http://
100 /var/lib/