intermittent ER_KEY_NOT_FOUND errors during insert

Bug #551250 reported by Mark Callaghan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
PBXT
Fix Committed
Undecided
Paul McCullagh

Bug Description

I am using MySQL 5.1.45 and PBXT 1.1 revision 851

my non-default pbxt settings are:
pbxt_index_cache_size=250M
pbxt_record_cache_size=750M
pbxt_flush_log_at_trx_commit=2

I am using a modified sysbench test (to be published real soon) in which 1 or more threads run nothing but the following on the sbtest table:
"INSERT INTO %s values(?,0,'0',"
           "'aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')"
           " ON DUPLICATE KEY UPDATE c = c + 1",

C has type 'char' in the table, I use it as an int.

I get intermittent errors:
ALERT: failed to execute MySQL query: `INSERT INTO sbtest values(1,0,'0','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeey
yyyyyyyyy') ON DUPLICATE KEY UPDATE c = c + 1`:
ALERT: Error 1032 Can't find record in 'sbtest'
FATAL: database error, exiting...

I run it in a loop: first for 1 thread, then 2, then 4, ... then 1024
The test was OK for 1 thread.
Then failed for 2, 4, 8, 16, 32 threads
Then was ok for 64 or more threads

There are no errors in the database error log.

Revision history for this message
Mark Callaghan (mdcallag) wrote :

Reproduction code is sysbench at https://code.launchpad.net/~mdcallag/sysbench/0.4-incupdate
vary the the value for --num-threads

sysbench --test=oltp --oltp-table-size=2000000 --max-time=60 --max-requests=0 --mysql-table-engine=pbxt \
  --db-ps-mode=disable --mysql-engine-trx=no --oltp-skip-trx --oltp-test-mode=incinsert \
  --oltp-dist-type=uniform --oltp-range-size=1 prepare

sysbench --test=oltp --oltp-table-size=2000000 --max-time=60 --max-requests=0 --mysql-table-engine=pbxt \
--db-ps-mode=disable --mysql-engine-trx=no --oltp-skip-trx --oltp-test-mode=incinsert \
--oltp-dist-type=uniform --oltp-range-size=1 --num-threads=1 --seed-rng=1 run

Revision history for this message
Paul McCullagh (paul-mccullagh) wrote :

Hi Mark,

Thanks for the bug report. We will check it out...

Changed in pbxt:
status: New → In Progress
Changed in pbxt:
assignee: nobody → Paul McCullagh (paul-mccullagh)
Revision history for this message
Paul McCullagh (paul-mccullagh) wrote :

I cannnot repeat this bug with PBXT 1.0.11. Most likely due to this bug fix:

RN318: Fixed a bug in the atomic R/W lock. This bug occurred on multi-core Linux when under extrem load. The affect was that an index lookup could fail. The index was not corrupted.

Revision history for this message
Paul McCullagh (paul-mccullagh) wrote :

I believe this bug is fixed in 1.0.11, which has already been pushed to 1.1, most like by this change in RN318.

Mark, if you can, please confirm the fix, and re-open the bug if this is not the case.

Changed in pbxt:
status: In Progress → Fix Committed
Revision history for this message
Mark Callaghan (mdcallag) wrote : Re: [Bug 551250] Re: intermittent ER_KEY_NOT_FOUND errors during insert

great, eventually I will do that. Our devel build now includes pbxt.

On Thu, May 27, 2010 at 9:00 AM, Paul McCullagh <
<email address hidden>> wrote:

> I believe this bug is fixed in 1.0.11, which has already been pushed to
> 1.1, most like by this change in RN318.
>
> Mark, if you can, please confirm the fix, and re-open the bug if this is
> not the case.
>
>
> ** Changed in: pbxt
> Status: In Progress => Fix Committed
>
> --
> intermittent ER_KEY_NOT_FOUND errors during insert
> https://bugs.launchpad.net/bugs/551250
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in PrimeBase XT: Fix Committed
>
> Bug description:
> I am using MySQL 5.1.45 and PBXT 1.1 revision 851
>
> my non-default pbxt settings are:
> pbxt_index_cache_size=250M
> pbxt_record_cache_size=750M
> pbxt_flush_log_at_trx_commit=2
>
> I am using a modified sysbench test (to be published real soon) in which 1
> or more threads run nothing but the following on the sbtest table:
> "INSERT INTO %s values(?,0,'0',"
> "'aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeeyyyyyyyyyy')"
> " ON DUPLICATE KEY UPDATE c = c + 1",
>
> C has type 'char' in the table, I use it as an int.
>
> I get intermittent errors:
> ALERT: failed to execute MySQL query: `INSERT INTO sbtest
> values(1,0,'0','aaaaaaaaaaffffffffffrrrrrrrrrreeeeeeeeeey
> yyyyyyyyy') ON DUPLICATE KEY UPDATE c = c + 1`:
> ALERT: Error 1032 Can't find record in 'sbtest'
> FATAL: database error, exiting...
>
> I run it in a loop: first for 1 thread, then 2, then 4, ... then 1024
> The test was OK for 1 thread.
> Then failed for 2, 4, 8, 16, 32 threads
> Then was ok for 64 or more threads
>
> There are no errors in the database error log.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/pbxt/+bug/551250/+subscribe
>

--
Mark Callaghan
<email address hidden>

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers