False positive testsuite failures with grep 2.10

[Laurynas Biveinis]

When grep 2.10 or newer (e.g. Ubuntu 12.04) is used, a part of tests fail with a false positive

+ grep 'xtrabackup: using O_DIRECT' /home/laurynas/percona/src/xtrabackup-page-filters/test/results/bug606981
grep: input file `/home/laurynas/percona/src/xtrabackup-page-filters/test/results/bug606981' is also the output
+ vlog 'xtrabackup was not using O_DIRECT for the input file.'
++ date '+%F %T'
++ basename t/bug606981.sh
+ echo '2012-05-08 13:22:21: bug606981.sh: xtrabackup was not using O_DIRECT for the input file.'
2012-05-08 13:22:21: bug606981.sh: xtrabackup was not using O_DIRECT for the input file.
+ exit -1

The "xtrabackup: using O_DIRECT" is present in the file, however grep refuses to run due to input and output being identical, this change is documented in 2.10 release notes:

  grep now rejects a command like "grep -r pattern . > out",
  in which the output file is also one of the inputs,
  because it can result in an "infinite" disk-filling loop.
  [bug present since "the beginning"]

[Alexey Kopytov (akopytov)]

We don't see such failures on Precise builds in Jenkins?

[Laurynas Biveinis (laurynas-biveinis)]

I did see these failures on Jenkins. Will provide links to examples soon (or confirm that the failures have disappeared since).

[Stewart Smith (stewart)]

On Wed, 13 Jun 2012 16:50:50 -0000, Alexey Kopytov <email address hidden> wrote:
> We don't see such failures on Precise builds in Jenkins?

Up until very recently we hadn't added precise to the XB jobs, this
could be why.

--
Stewart Smith

[Alexey Kopytov (akopytov)]

On 14.06.12 4:58, Stewart Smith wrote:
> On Wed, 13 Jun 2012 16:50:50 -0000, Alexey Kopytov <email address hidden> wrote:
>> We don't see such failures on Precise builds in Jenkins?
>
> Up until very recently we hadn't added precise to the XB jobs, this
> could be why.
>

OK, Precise builders were added yesterday and now show the problem:
http://jenkins.percona.com/view/XtraBackup/job/percona-xtrabackup-2.0/921/

[Alexey Kopytov (akopytov)]

It turns out the failures are a result of a bug in grep itself. All failing tests except 2 do the correct thing: they use -q which supresses the output, so grepping the same file where the output is redirected is not a problem, because grep itself generates no output.

However, grep version shipped with Ubuntu 12.04 is buggy and fails even when -q is specified. The bug was fixed in grep 2.11:

" grep no longer rejects "grep -qr . > out", i.e., when run with -q
  and an input file is the same as the output file, since with -q
  grep generates no output, so there is no risk of infinite loop or
  of an output-affecting race condition. Thus, the use of the following
  options also disables the input-equals-output failure:
    --max-count=N (-m) (for N >= 2)
    --files-with-matches (-l)
    --files-without-match (-L)
  [bug introduced in grep-2.10]"

So fixing those 2 tests that do not use -q will not help, as all tests using grep will still be failing on Ubuntu 12.04. The workaround is to redirect the output to /dev/null.

[Laurynas Biveinis (laurynas-biveinis)]

A note to self with affected tests on the current 2.1 trunk:
t/bug606981.sh t/bug759225.sh t/bug766033.sh t/bug972169.sh t/bug999750.sh t/ib_empty_dir.sh t/xb_stats.sh