if statement with pipe to grep randomly fails if pipefail is set

[Michael Kerrin]

The following bash script will fail randomly on trusty but when tested on saucy it worked all the time. On the system we test we have kvm installed and the if statement should evaluate to false and we should see ten PASS lines and no Error. And this should happen all the time.

======
#!/bin/bash

set -eu
set -o pipefail

if ! lsmod | grep -q -e ^kvm_intel -e ^kvm_amd ; then
    echo 'Error: check that virtualisation is enabled in your BIOS'
else
    echo "PASS"
fi
=======

As can be seen if I run it 10 times I get 2 PASS and 8 failures

=======
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ for i in $(seq 0 9) ; do bash test-simple.bash ; done
PASS
PASS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
Error: check that virtualisation is enabled in your BIOS
=======

Now if I turn off the pipefail then the script passes all the time and just prints out 10 PASS lines.

The script uses bash, grep and lsmod and I have the following versions on trusty of them

kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ dpkg -l | grep bash
ii bash 4.3-7ubuntu1.5 amd64 GNU Bourne Again SHell
ii bash-completion 1:2.1-4 all programmable completion for the bash shell
ii command-not-found 0.3ubuntu12 all Suggest installation of packages in interactive bash sessions
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ dpkg -l | grep grep
ii grep 2.16-1 amd64 GNU grep, egrep and fgrep
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ dpkg -l | grep kmod
ii kmod 15-0ubuntu6 amd64 tools for managing Linux kernel modules
ii libkmod2:amd64 15-0ubuntu6 amd64 libkmod shared library
ii module-init-tools 15-0ubuntu6 all transitional dummy package (module-init-tools to kmod)

[Michael Kerrin (michael-kerrin-w)]

We can also see the following result:

errin@kerrin-HP-Z620-Workstation:~/ce_build$ lsmod | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
141
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ lsmod | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
141
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ lsmod | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
141
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ lsmod | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
0
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ lsmod | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
141
kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ lsmod | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
141

One out of a lot passed.

[Michael Kerrin (michael-kerrin-w)]

But this seems to work

kerrin@kerrin-HP-Z620-Workstation:~/ce_build$ echo "kvm_intel xxx" | grep -q -e ^kvm_amd -e ^kvm_intel; echo $?
0

[xhienne (xhienne)]

This is not a bug, although some might disagree with the current behavior of bash.

With a command like 'yes | grep -q .', because the grep terminates early and the yes is writing past the end of the pipe buffer, the yes process is _always_ terminated with a SIGPIPE signal (which translates to a return value of 141). Combined with the pipefail option, the whole command is considered to have failed.

pipefail should be used scarcely, that's why it is not enabled by default.