test suite failures

[Pavel Stratil]

I have opened the topic of failing tests in 'answers' but the topic remains open for too long (https://answers.launchpad.net/drizzle/+question/114414). A working testsuite is important for packaging drizzle on gentoo linux - it allows devs to catch runtime errors. Without it, drizzle releases are held back without reason. Follows a summary, more details in the 'answers' link above.

Stopping All Servers
Failed 15/332 tests, 95.48% were successful.

The log files in var/log may give you some hint
of what went wrong.
If you want to report this error, go to:
 http://bugs.launchpad.net/drizzle
The servers were restarted 48 times
Spent 2028.406 of 3279 seconds executing testcases

mysql-test-run in default mode: *** Failing the test(s): main.connect main.data_dictionary_like_info main.drizzleslap main.information_schema main.lowercase_fs_off simple_user_policy.basic registry_dictionary.data_dictionary registry_dictionary.slap mysql_protocol.prototest logging_stats.command logging_stats.cumulative logging_stats.max_session logging_stats.max_user auth_test.basic auth_file.basic

[Monty Taylor (mordred)]

On 06/25/2010 04:31 PM, Patrick Crews wrote:
> ** Changed in: drizzle
> Assignee: (unassigned) => Patrick Crews (patrick-crews)
>

I would really like to know what's going on here, and if there is
something in the reported build environment that we can replicate in our
hudson setup. We have a gentoo builder, and nothing makes it to trunk
with failing test cases, but obviously there is an issue here...

I'd love any suggestions of how to make our testing of this better.

[Pavel Stratil (pavel-senbonzakura)]

Monty, I suppose that its not related to the build environment but to plugins built. This fails tests

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64 --disable-all --disable-static --disable-dependency-tracking --disable-mtmalloc --enable-tcmalloc --enable-libmemcached --enable-libgearman --enable-libldap --with-auth-http-plugin --with-auth-pam-plugin --with-md5-plugin --with-gearman_udf-plugin --with-logging_gearman-plugin --with-memcache-functions-plugins --with-auth-ldap-plugin --with-logging-stats --without-hello-world-plugin --disable-pbxt-plugin --without-pbxt-plugin --disable-rabbitmq-plugin --without-rabbitmq-plugin --disable-embedded-innodb-plugin --without-embedded-innodb-plugin --enable-auth-test-plugin --with-auth-test-plugin --enable-auth-file-plugin --with-auth-file-plugin --enable-simple-user-policy-plugin --with-simple-user-policy-plugin --enable-logging-stats-plugin --with-logging-stats-plugin --enable-logging-syslog-plugin --with-logging-syslog-plugin

This not:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64 --disable-all --disable-static --disable-dependency-tracking --disable-mtmalloc --enable-tcmalloc --enable-libmemcached --enable-libgearman --with-auth-http-plugin --with-auth-pam-plugin --with-md5-plugin --with-gearman_udf-plugin --with-logging_gearman-plugin --with-memcache_functions-plugins --with-logging_stats --without-hello-world-plugin --disable-pbxt-plugin --without-pbxt-plugin --disable-rabbitmq-plugin --without-rabbitmq-plugin --disable-embedded-innodb-plugin --without-embedded-innodb-plugin --disable-auth-ldap-plugin --disable-libldap --without-auth-ldap-plugin

[Monty Taylor (mordred)]

Wonderful! That's very helpful.

On 06/26/2010 06:50 AM, Pavel Stratil wrote:
> Monty, I suppose that its not related to the build environment but to
> plugins built. This fails tests
>
> ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-
> linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info
> --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib
> --libdir=/usr/lib64 --disable-all --disable-static --disable-dependency-
> tracking --disable-mtmalloc --enable-tcmalloc --enable-libmemcached
> --enable-libgearman --enable-libldap --with-auth-http-plugin --with-
> auth-pam-plugin --with-md5-plugin --with-gearman_udf-plugin --with-
> logging_gearman-plugin --with-memcache-functions-plugins --with-auth-
> ldap-plugin --with-logging-stats --without-hello-world-plugin --disable-
> pbxt-plugin --without-pbxt-plugin --disable-rabbitmq-plugin --without-
> rabbitmq-plugin --disable-embedded-innodb-plugin --without-embedded-
> innodb-plugin --enable-auth-test-plugin --with-auth-test-plugin
> --enable-auth-file-plugin --with-auth-file-plugin --enable-simple-user-
> policy-plugin --with-simple-user-policy-plugin --enable-logging-stats-
> plugin --with-logging-stats-plugin --enable-logging-syslog-plugin
> --with-logging-syslog-plugin
>
> This not:
>
> ./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-
> linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info
> --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib
> --libdir=/usr/lib64 --disable-all --disable-static --disable-dependency-
> tracking --disable-mtmalloc --enable-tcmalloc --enable-libmemcached
> --enable-libgearman --with-auth-http-plugin --with-auth-pam-plugin
> --with-md5-plugin --with-gearman_udf-plugin --with-logging_gearman-
> plugin --with-memcache_functions-plugins --with-logging_stats --without-
> hello-world-plugin --disable-pbxt-plugin --without-pbxt-plugin
> --disable-rabbitmq-plugin --without-rabbitmq-plugin --disable-embedded-
> innodb-plugin --without-embedded-innodb-plugin --disable-auth-ldap-
> plugin --disable-libldap --without-auth-ldap-plugin
>

[Monty Taylor (mordred)]

On 06/26/2010 06:50 AM, Pavel Stratil wrote:
> Monty, I suppose that its not related to the build environment but to
> plugins built. This fails tests
>

Here are the differences:

Failing options:

--enable-libldap --with-auth-ldap-plugin --enable-auth-test-plugin
--with-auth-test-plugin
--enable-auth-file-plugin --with-auth-file-plugin
--enable-simple-user-policy-plugin --with-simple-user-policy-plugin
--enable-logging-stats-plugin
--with-logging-stats-plugin --enable-logging-syslog-plugin
--with-logging-syslog-plugin

Passing:

--disable-auth-ldap-plugin --disable-libldap --without-auth-ldap-plugin

[Joe Daly (skinny.moey)]

The logging_stats plugin changes users, in one test theres 500 users if you had the auth-file-plugin enabled it would definately fail. I suspect the other tests are a similiar failure.

[Monty Taylor (mordred)]

Aha! Good catch there joe.

The --enable options to configure change the list of default plugins. I'm not sure it's feasible to run the test suite with optional plugins enabled by default. The tests in the suite for optional plugins configure and set up the server properly for having that plugin enabled.

Would it be possible in the gentoo build to remove the extra --enalbe options and only use --with instead?

"Joe Daly" <email address hidden> wrote:

>The logging_stats plugin changes users, in one test theres 500 users if
>you had the auth-file-plugin enabled it would definately fail. I suspect
>the other tests are a similiar failure.
>
>--
>test suite failures
>https://bugs.launchpad.net/bugs/598659
>You received this bug notification because you are a member of Drizzle-
>developers, which is subscribed to Drizzle.
>
>Status in A Lightweight SQL Database for Cloud and Web: New
>Status in Gentoo Linux: New
>
>Bug description:
>I have opened the topic of failing tests in 'answers' but the topic remains open for too long (https://answers.launchpad.net/drizzle/+question/114414).. A working testsuite is important for packaging drizzle on gentoo linux - it allows devs to catch runtime errors. Without it, drizzle releases are held back without reason. Follows a summary, more details in the 'answers' link above.
>
>Stopping All Servers
>Failed 15/332 tests, 95.48% were successful.
>
>The log files in var/log may give you some hint
>of what went wrong.
>If you want to report this error, go to:
> http://bugs.launchpad.net/drizzle
>The servers were restarted 48 times
>Spent 2028.406 of 3279 seconds executing testcases
>
>mysql-test-run in default mode: *** Failing the test(s): main.connect main.data_dictionary_like_info main.drizzleslap main.information_schema main.lowercase_fs_off simple_user_policy.basic registry_dictionary.data_dictionary registry_dictionary.slap mysql_protocol.prototest logging_stats.command logging_stats.cumulative logging_stats.max_session logging_stats.max_user auth_test.basic auth_file.basic
>
>
>
>
>

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Patrick Crews (patrick-crews)]

Definitely a good catch. Thanks for looking at this.
I'll make a note about this in our wiki as well - working on improving wiki
docs for testing.

On Sat, Jun 26, 2010 at 10:11 AM, Monty Taylor <email address hidden> wrote:

> Aha! Good catch there joe.
>
> The --enable options to configure change the list of default plugins.
> I'm not sure it's feasible to run the test suite with optional plugins
> enabled by default. The tests in the suite for optional plugins
> configure and set up the server properly for having that plugin enabled.
>
> Would it be possible in the gentoo build to remove the extra --enalbe
> options and only use --with instead?
>
> "Joe Daly" <email address hidden> wrote:
>
> >The logging_stats plugin changes users, in one test theres 500 users if
> >you had the auth-file-plugin enabled it would definately fail. I suspect
> >the other tests are a similiar failure.
> >
> >--
> >test suite failures
> >https://bugs.launchpad.net/bugs/598659
> >You received this bug notification because you are a member of Drizzle-
> >developers, which is subscribed to Drizzle.
> >
> >Status in A Lightweight SQL Database for Cloud and Web: New
> >Status in Gentoo Linux: New
> >
> >Bug description:
> >I have opened the topic of failing tests in 'answers' but the topic
> remains open for too long (
> https://answers.launchpad.net/drizzle/+question/114414).. A working
> testsuite is important for packaging drizzle on gentoo linux - it allows
> devs to catch runtime errors. Without it, drizzle releases are held back
> without reason. Follows a summary, more details in the 'answers' link above.
> >
> >Stopping All Servers
> >Failed 15/332 tests, 95.48% were successful.
> >
> >The log files in var/log may give you some hint
> >of what went wrong.
> >If you want to report this error, go to:
> > http://bugs.launchpad.net/drizzle
> >The servers were restarted 48 times
> >Spent 2028.406 of 3279 seconds executing testcases
> >
> >mysql-test-run in default mode: *** Failing the test(s): main.connect
> main.data_dictionary_like_info main.drizzleslap main.information_schema
> main.lowercase_fs_off simple_user_policy.basic
> registry_dictionary.data_dictionary registry_dictionary.slap
> mysql_protocol.prototest logging_stats.command logging_stats.cumulative
> logging_stats.max_session logging_stats.max_user auth_test.basic
> auth_file.basic
> >
> >
> >
> >
> >
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>
> --
> test suite failures
> https://bugs.launchpad.net/bugs/598659
> You received this bug notification because you are a bug assignee.
>
> Status in A Lightweight SQL Database for Cloud and Web: New
> Status in Gentoo Linux: New
>
> Bug description:
> I have opened the topic of failing tests in 'answers' but the topic remains
> open for too long (https://answers.launchpad.net/drizzle/+question/114414).
> A working testsuite is important for packaging drizzle on gentoo linux - it
> allows devs to catch runtime errors. Without it, drizzle releases are held
> back without reason. Follows a summary, more details in the 'answers' link
> above.
>
> Stopping All Servers
> Failed 15/332 tests, 95.48% were successful.
>
> T...

[Pavel Stratil (pavel-senbonzakura)]

Would it be possible in the gentoo build to remove the extra --enalbe options and only use --with instead?

should be possible! I'll give it a spin and let you know...

[Pavel Stratil (pavel-senbonzakura)]

Managed to reduce the number of failed tests to two. Here's the configure:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64 --disable-static --disable-dependency-tracking --disable-mtmalloc --enable-tcmalloc --enable-libmemcached --enable-libgearman --enable-libldap --with-auth-http-plugin --with-auth-pam-plugin --with-md5-plugin --with-gearman_udf-plugin --with-logging_gearman-plugin

mysql-test-run in default mode: *** Failing the test(s): main.drizzleslap transaction_log.slap
mysql-test-run: *** ERROR: there were failing test cases

I'm building drizzle on a laptop so it will take hours before I can verify if this relates to --enable-libldap. Does anyone know if these are related? What's the current state of ldap support/plugins in drizzle?

Thanks!

[Lee Bieber (kalebral-deactivatedaccount)]

The ldap plugin is supported, we have a regular build testing ldap in our Hudson infrastructure - http://hudson.drizzle.org/job/drizzle-build-test-ldap/

[Patrick Crews (patrick-crews)]

I'm currently trying to duplicate on my machine. Could you include the test-run failure output for those two tests you have noted above?

[Pavel Stratil (pavel-senbonzakura)]

for now:

main.drizzleslap [ fail ]
timeout

Stopping All Servers
mysql-test-run: WARNING: Forcing kill of process 29271
Restoring snapshot of databases
Resuming Tests

...

transaction_log.slap [ fail ]
drizzletest: At line 18: query 'SELECT COUNT(*) FROM test.t1' failed: 20: drizzle_state_read:lost connection to server (EOF)

The result from queries just before the failure was:
SET GLOBAL transaction_log_truncate_debug= true;
CREATE SCHEMA drizzleslap;
CREATE TABLE t1
(
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
rand_float FLOAT NOT NULL
) ENGINE=InnoDB;
SELECT COUNT(*) FROM test.t1;

More results from queries before failure can be found in /var/tmp/portage/dev-db/drizzle-2010.06.1634/work/drizzle-2010.06.1634/tests/var/log/slap.log

... can give you the slap.log when I get there. compiling again ...

[Pavel Stratil (pavel-senbonzakura)]

hmm, i thought that the slap could be somehow connected to ldap since i didnt get this failure with an old ebuild which had libldap disabled. so i disabled it again and this time got main.drizzleslap [ fail ] timeout again.

[Andrew Hutchings (linuxjedi)]

Marking as Fix Released as it was confirmed in the Question that this now resolved.