False positive testsuite failures with grep 2.10

Reported by Laurynas Biveinis on 2012-05-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona XtraBackup
High
Alexey Kopytov
1.6
High
Alexey Kopytov
2.0
High
Alexey Kopytov
2.1
High
Alexey Kopytov

Bug Description

When grep 2.10 or newer (e.g. Ubuntu 12.04) is used, a part of tests fail with a false positive

+ grep 'xtrabackup: using O_DIRECT' /home/laurynas/percona/src/xtrabackup-page-filters/test/results/bug606981
grep: input file `/home/laurynas/percona/src/xtrabackup-page-filters/test/results/bug606981' is also the output
+ vlog 'xtrabackup was not using O_DIRECT for the input file.'
++ date '+%F %T'
++ basename t/bug606981.sh
+ echo '2012-05-08 13:22:21: bug606981.sh: xtrabackup was not using O_DIRECT for the input file.'
2012-05-08 13:22:21: bug606981.sh: xtrabackup was not using O_DIRECT for the input file.
+ exit -1

The "xtrabackup: using O_DIRECT" is present in the file, however grep refuses to run due to input and output being identical, this change is documented in 2.10 release notes:

  grep now rejects a command like "grep -r pattern . > out",
  in which the output file is also one of the inputs,
  because it can result in an "infinite" disk-filling loop.
  [bug present since "the beginning"]

Stewart Smith (stewart) on 2012-06-13
Changed in percona-xtrabackup:
importance: Undecided → High
Alexey Kopytov (akopytov) wrote :

We don't see such failures on Precise builds in Jenkins?

Changed in percona-xtrabackup:
status: Confirmed → Incomplete

I did see these failures on Jenkins. Will provide links to examples soon (or confirm that the failures have disappeared since).

On Wed, 13 Jun 2012 16:50:50 -0000, Alexey Kopytov <email address hidden> wrote:
> We don't see such failures on Precise builds in Jenkins?

Up until very recently we hadn't added precise to the XB jobs, this
could be why.

--
Stewart Smith

Alexey Kopytov (akopytov) wrote :

On 14.06.12 4:58, Stewart Smith wrote:
> On Wed, 13 Jun 2012 16:50:50 -0000, Alexey Kopytov <email address hidden> wrote:
>> We don't see such failures on Precise builds in Jenkins?
>
> Up until very recently we hadn't added precise to the XB jobs, this
> could be why.
>

OK, Precise builders were added yesterday and now show the problem:
http://jenkins.percona.com/view/XtraBackup/job/percona-xtrabackup-2.0/921/

It turns out the failures are a result of a bug in grep itself. All failing tests except 2 do the correct thing: they use -q which supresses the output, so grepping the same file where the output is redirected is not a problem, because grep itself generates no output.

However, grep version shipped with Ubuntu 12.04 is buggy and fails even when -q is specified. The bug was fixed in grep 2.11:

" grep no longer rejects "grep -qr . > out", i.e., when run with -q
  and an input file is the same as the output file, since with -q
  grep generates no output, so there is no risk of infinite loop or
  of an output-affecting race condition. Thus, the use of the following
  options also disables the input-equals-output failure:
    --max-count=N (-m) (for N >= 2)
    --files-with-matches (-l)
    --files-without-match (-L)
  [bug introduced in grep-2.10]"

So fixing those 2 tests that do not use -q will not help, as all tests using grep will still be failing on Ubuntu 12.04. The workaround is to redirect the output to /dev/null.

summary: - False positive testsuite failures with grep 2.10+
+ False positive testsuite failures with grep 2.10

A note to self with affected tests on the current 2.1 trunk:
t/bug606981.sh t/bug759225.sh t/bug766033.sh t/bug972169.sh t/bug999750.sh t/ib_empty_dir.sh t/xb_stats.sh

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers