Sporadic "buf_LRU_old_ratio >= 51" assertion failures in 5.1 debug builds in Jenkins

Bug #924492 reported by Patrick Crews
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Percona XtraBackup moved to https://jira.percona.com/projects/PXB
Fix Released
Low
Alexey Kopytov
2.0
Fix Released
Low
Alexey Kopytov
2.1
Fix Released
Low
Alexey Kopytov

Bug Description

While the xb_stats test passes fine when using the Sakila database test bed, when using the percona.zz randgen test bed, we see a crash with the following error:
http://jenkins.percona.com/view/Percona%20Xtrabackup/job/percona-xtrabackup-trunk-kewpie/lastCompletedBuild/BUILD_TYPE=debug,Host=debian6-32,xtrabackuptarget=innodb51/testReport/%28root%29/xtrabackup_main/xb_stats_test/

<snip>
  t120131 12:35:52 InnoDB: Assertion failure in thread 1075345088 in file buf/buf0lru.c line 908
InnoDB: Failing assertion: buf_LRU_old_ratio >= 51
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
able: test/C, index: col_enum_key, space id: 0, root page: 211, zip size: 0
  estimated statistics in dictionary:
    key vals: 1, leaf pages: 1, size pages: 1
  real statistics:
        leaf pages: recs=1, pages=1, data=11 bytes, data/pages=0%
<snip>
table: test/CC, index: PRIMARY, space id: 0, root page: 405, zip size: 0
  estimated statistics in dictionary:
    key vals: 95, leaf pages: 800, size pages: 929
  real statistics:
     level 1 pages: pages=1, data=615 bytes, data/pages=3%
        leaf pages: Aborted

I am not certain what the key difference is between datasets, beyond use of blobs & random data.

To repeat. From the attached branch:
./kewpie.py --libeatmydata --force xb_stats_sakila_test xb_stats_test
<snip>
20120131-152109 INFO STATUS: 1
20120131-152231 ===============================================================
20120131-152231 TEST NAME [ RESULT ] TIME (ms)
20120131-152231 ===============================================================
20120131-152231 xtrabackup_main.xb_stats_sakila_test [ pass ] 79201
20120131-152237 xtrabackup_main.xb_stats_test [ fail ] 3259
20120131-152237 test_xb_stats (xb_stats_test.basicTest) ... FAIL
20120131-152237
20120131-152237 ======================================================================
20120131-152237 FAIL: test_xb_stats (xb_stats_test.basicTest)
20120131-152237 ----------------------------------------------------------------------

Related branches

Revision history for this message
Patrick Crews (patrick-crews) wrote :

Tested on Ubuntu Lucid 64 bit myself and appears to be failing on innodb51 builds

Changed in percona-xtrabackup:
status: New → Confirmed
Revision history for this message
Patrick Crews (patrick-crews) wrote :

This appears to be failing during the prepare phase of working with the backup.
I am seeing a similar failure on bug514068_test during prepare phase. As with this test, experiments with other data sets allowed the test to pass without issue. - Bug#924517

Revision history for this message
Patrick Crews (patrick-crews) wrote :

The test data consists of tables of similar composition. Each table has the same columns, but they are created in randomized order. They are populated with 0,0,1,1,10,10,100,100 rows each.

CREATE TABLE `CC` (
`col_enum_not_null_key` enum ('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') not null,
`col_bigint_not_null` bigint not null,
`col_enum` enum ('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'),
`col_char_10_not_null_key` char(10) not null,
`col_char_255_key` char(255),
`col_char_10` char(10),
`col_text_key` text,
`col_bigint_not_null_key` bigint not null,
`col_text_not_null_key` text not null,
`col_char_255_not_null` char(255) not null,
`col_enum_key` enum ('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'),
`col_char_10_not_null` char(10) not null,
`col_int_key` int,
`col_char_255` char(255),
`col_text_not_null` text not null,
`col_char_255_not_null_key` char(255) not null,
`col_bigint_key` bigint,
pk integer auto_increment,
`col_enum_not_null` enum ('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') not null,
`col_int_not_null` int not null,
`col_char_10_key` char(10),
`col_bigint` bigint,
`col_int_not_null_key` int not null,
`col_int` int,
`col_text` text,
/*Indices*/
key (`col_enum_not_null_key` ),
key (`col_char_10_not_null_key` ),
key (`col_char_255_key` ),
key (`col_text_key` (255)),
key (`col_bigint_not_null_key` ),
key (`col_text_not_null_key` (255)),
key (`col_enum_key` ),
key (`col_int_key` ),
key (`col_char_255_not_null_key` ),
key (`col_bigint_key` ),
primary key (pk),
key (`col_char_10_key` ),
key (`col_int_not_null_key` )) ;

Revision history for this message
Patrick Crews (patrick-crews) wrote :

Appears to be failing due to indexed text column:
Changing the percona.zz file like this (or by removing 'text' form the types list will allow the test to pass.

=== modified file 'kewpie/randgen/conf/percona/percona.zz'
--- old/kewpie/randgen/conf/percona/percona.zz 2011-10-13 20:38:47 +0000
+++ new/kewpie/randgen/conf/percona/percona.zz 2012-02-01 11:31:39 +0000
@@ -30,7 +30,7 @@ $tables = {

 $fields = {
         types => [ 'bigint' , 'int', 'char(10)', 'char(255)', 'enum' , 'text' ],
- indexes => [undef, 'key' ],
+ indexes => [undef ],
         null => [undef, 'not null' ],
         default => [undef, 'default null' ]
 };

Revision history for this message
Alexey Kopytov (akopytov) wrote :
Revision history for this message
Stewart Smith (stewart) wrote :

so... it looks like that kewpie tests are now about on par with old test suite? That is, we should look at running both in main test run and gradually replace old suite?

Revision history for this message
Alexey Kopytov (akopytov) wrote :

There are other kewpie-specific failures that we have been discussing with Patrick. But we can look into running both as soon as kewpie gets reintegrated after test cases review.

Stewart Smith (stewart)
Changed in percona-xtrabackup:
status: Confirmed → Triaged
importance: Undecided → High
Revision history for this message
Alexey Kopytov (akopytov) wrote :

Setting priority to Low, as the problem seems to be benign. The reason is that InnoDB initializes buf_LRU_old_ratio, but XtraBackup does not. So the first call to buf_LRU_old_adjust_len() triggers the debug-only assertion. It is not normally called by XtraBackup in the test suite (requires a certain level of pages in the buffer pool).

So it does not indicate any user-affecting problem and is only visible in debug builds. But we still have to fix debug builds in Jenkins.

summary: - test xb_stats failing in debug builds with different data set than
- original test suite
+ Sporadic "buf_LRU_old_ratio >= 51" assertion failures in 5.1 debug
+ builds in Jenkins
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-843

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.