Comment 21 for bug 673105

w00t! You are the man! 12 hour days and randgen crash courses paid off ; )

On Fri, Dec 17, 2010 at 4:09 PM, David Shrewsbury <<email address hidden>
> wrote:

> Savepoints were lingering and/or not setup properly by the engines
> because the transaction was not started properly when a SAVEPOINT (or
> even ROLLBACK TO SAVEPOINT) implicitly begins a new transaction with
> AUTOCOMMIT=OFF.
>
> If an implicit transaction is started by a statement that touches
> tables, the call to startStatement() from lockTables() would
> automatically start a transaction if one was needed. Since savepoints
> don't go through this code, we needed checks to look for this case.
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/673105
>
> Title:
> Drizzle transaction log showing differences from server state in
> concurrent testing
>
> Status in A Lightweight SQL Database for Cloud Infrastructure and Web
> Applications:
> In Progress
>
> Bug description:
> When testing Drizzle's transaction log in a concurrent environment with
> the randgen, we are seeing differences between the server's state and the
> SQL captured in the transaction log. The failures appear to be the results
> of INSERT's, UPDATE's, and DELETE's.
>
> The test consists of executing several queries (in several threads) against
> the server. We then generate SQL from the transaction log and use the SQL
> to populate a validation server. We compare Drizzledump output between
> 'master' and 'slave' to validate.
>
> The failure is generally consistent, but appears to be subject to timing
> issues. The randgen fork()'s off processes for each 'thread' used and the
> exact timing and interaction between threads varies. However, we see
> failures at certain thread/query_count combinations:
>
> 2 threads, 2500 queries
> 5 threads, 70-75 queries
> 10 thread, 50 queries.
>
> This test grammar was designed to produce a higher percentage of valid
> queries and tables can easily grow quite large.
>
> At present, I am not certain what is causing the issue, but it is an
> interaction between connections. These failures are not repeatable in
> single-thread scenarios.
>
> TO REPEAT:
> 1) Start a drizzle server to test, enable the transaction log
> ./test-run --start-and-exit --drizzled=--transaction-log.enable (expected
> port 9306) --basedir=$PWD/..
> 2) Start a second server to validate (expected port 9307)
> 3) Run the randgen and observe the results. Command line:
> perl ./gentest.pl \
> --dsn=dbi:drizzle:host=localhost:port=9306:user=root:password="":database=test
> \
> --gendata=conf/drizzle/translog_drizzle.zz
> --grammar=conf/drizzle/translog_concurrent3.yy \
> --queries=70 --threads=5 --sqltrace --debug
> --Reporter=DrizzleConcurrentTransactionLog
>
> Sample output:
> # 2010-11-09T11:51:18 Executing diff --unified
> /tmp//translog_rpl_dump_4869_9306.sql /tmp//translog_rpl_dump_4869_9307.sql
> --- /tmp//translog_rpl_dump_4869_9306.sql 2010-11-09
> 11:51:17.981424189 -0500
> +++ /tmp//translog_rpl_dump_4869_9307.sql 2010-11-09
> 11:51:18.491212405 -0500
> @@ -1014,21 +1014,46 @@
> INSERT INTO `a` VALUES
> (NULL,'a','u',6,'7073747639715495936','b','','GHGTD',8,'you','mean',1143,'b','RZARM','UUDDE','UKVQC','','','1147292005072633856','BSUXB',6,8,'there','',NULL);
> INSERT INTO `a` VALUES
> (NULL,'q','z',6,'7891995397013372928','e','','JFLJB',NULL,'okay','SALOT',1144,'t','ZPJCL','z','NMCDY','','','7','want',1,1,'TLLTD','',NULL);
> INSERT INTO `a` VALUES
> (NULL,'q','z',6,'7891995397013372928','e','','JFLJB',NULL,'okay','SALOT',1145,'t','ZPJCL','z','NMCDY','','','7','want',1,1,'TLLTD','',NULL);
> +INSERT INTO `a` VALUES
> ('-7326793643778375680','h','m',1383792640,'6322209451898044416','v','','MZTZG',NULL,'will','he',1146,'u','MYQYS','I','that\'s','','','-7835700401671241728','k',6,3,'HMCNM','','6');
> +INSERT INTO `a` VALUES
> ('-6968194523448999936','i','t',-1211891712,'-6355704974126612480','o','','j',NULL,'LQVCJ','we',1147,'c','could','MWBTI','I\'ll','','','3','that',0,-988217344,'CCQZH','','8');
> INSERT INTO `a` VALUES
> (NULL,'q','z',6,'7891995397013372928','e','','JFLJB',NULL,'okay','SALOT',1148,'t','ZPJCL','z','NMCDY','','','7','want',1,126,'TLLTD','',NULL);
> +INSERT INTO `a` VALUES
> ('-6968194523448999936','i','t',-1211891712,'-6355704974126612480','o','','j',NULL,'LQVCJ','we',1149,'c','could','MWBTI','I\'ll','','','3','that',0,-988217344,'CCQZH','','8');
> +INSERT INTO `a` VALUES
> ('-6882063180575539200','v','q',NULL,'6230167134513659904','e','','who',-351141888,'t','k',1150,'x','yes','l','s','','','8706302504637300736','y',-1453522944,5,'NTVJG','','287104476244869120');
>
>
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/drizzle/+bug/673105/+subscribe
>