When installing 5.2 from debian packages, debian-sys-maint user is misconfigured

[Walter Heck]

When installing mariadb 5.2 from downloaded .deb packages on debian lenny, there's a few problems i run into. I'm not sure which causes what, but here's a small list. Terminal output is attached.

* There's a message about debian-sys-maint access denied.
* There's a message that Innodb is disabled, yet it shows up in a show engines statement.
* the default config file contains options that generate warnings on server start
* The server won't properly shutdown.

[Kristian Nielsen (knielsen)]

1. The debian-sys-maint account should be created during install, it is used internally by the postinst scripts. My guess is that something went wrong with this in the many failed install attempts. It is a huge problem that we do not have apt-get'able package repositories at the moment, packages need to be installed in a specific order, which is _very_ hard to get right manually.

You may have to create that user manually (or remove --purge + reinstall in right order should also work, if you haven't yet put data in).

2. When the postinst script installs the initial database, it runs mysqld with special options that disable InnoDB (to reduce risk of failing to install due to problems with innodb options), this is nothing to worry about (agree it is a bit confusing to see these in the logs, and also the warnings about --log-xxx options).

3. The failure to shutdown is a consequence of the problem with the debian-sys-maint account.

Looking at the log about the problem with debian-sys-maint ... It says 'using password: NO'. This suggests something wrong with the file cat /etc/mysql/debian.cnf. Can you show the contents of this file, with any password replaced by XXX ? It should look something like this:

[client]
host = localhost
user = debian-sys-maint
password = XXX
socket = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
user = debian-sys-maint
password = XXX
socket = /var/run/mysqld/mysqld.sock
basedir = /usr

If this looks ok, you can check that `mysqladmin --defaults-file=/etc/mysql/debian.cnf ping` works; if not then maybe the account inside mariadb is wrong/missing (but the Using password: NO suggests a problem with debian.cnf)

Once we understand better what the problem is, we can try see if the install procedure can be made more robust to not lose the debian-sys-maint account in case of install failure

(It is also possible that an existing /etc/mysql/debian.cnf file with wrong content could have caused this; seeing the content of the file should help determine this).

[Walter Heck (walterheck)]

On Fri, Nov 12, 2010 at 17:43, Kristian Nielsen
<email address hidden> wrote:
> 1. The debian-sys-maint account should be created during install, it is
> used internally by the postinst scripts. My guess is that something went
> wrong with this in the many failed install attempts. It is a huge
> problem that we do not have apt-get'able package repositories at the
> moment, packages need to be installed in a specific order, which is
> _very_ hard to get right manually.
I was sitting next to Colin while doing the install and he warned me
about it actually. I went ahead and did it anyway, cos even though
dpkg sucks for dependencies like this, it shouldn't actually cause
mayhem like this :)
Also, i'll talk to Arjen some more to put the pressure on getting the
OurDelta repo's in shape.

> You may have to create that user manually (or remove --purge + reinstall
> in right order should also work, if you haven't yet put data in).
I'll create manually. This is a test VM for http://fossasia.org, so
i'm not fussed about messign with it :)

> 2. When the postinst script installs the initial database, it runs
> mysqld with special options that disable InnoDB (to reduce risk of
> failing to install due to problems with innodb options), this is nothing
> to worry about (agree it is a bit confusing to see these in the logs,
> and also the warnings about --log-xxx options).
Agreed.

> 3. The failure to shutdown is a consequence of the problem with the
> debian-sys-maint account.
Figured that.

> Looking at the log about the problem with debian-sys-maint ... It says
> 'using password: NO'. This suggests something wrong with the file cat
> /etc/mysql/debian.cnf. Can you show the contents of this file, with any
> password replaced by XXX ? It should look something like this:
db:~# cat /etc/mysql/debian.cnf
# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host = localhost
user = debian-sys-maint
password = yomamma
socket = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host = localhost
user = debian-sys-maint
password = yomamma
socket = /var/run/mysqld/mysqld.sock
basedir = /usr

> If this looks ok, you can check that `mysqladmin --defaults-
> file=/etc/mysql/debian.cnf ping` works; if not then maybe the account
> inside mariadb is wrong/missing (but the Using password: NO suggests a
> problem with debian.cnf)
The account is missing:
db:~# mysqladmin --defaults-file=/etc/mysql/debian.cnf ping
mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using
password: NO)'

MariaDB [(none)]> select user, host, password from mysql.user;
+------+----------------+-------------------------------------------+
| user | host | password |
+------+----------------+-------------------------------------------+
| root | localhost | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
| root | db.example.com | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
| root | 127.0.0.1 | *81F5E21E35407D884A6CD4A731AEBFB6AF209E1B |
+------+----------------+-------------------------------------------+
3 rows in set (0.00 sec)

> (It is also possible that...

[Walter Heck (walterheck)]

Adding the debian-sys-maint user by hand fixed most of my problems:
==== Maria commands
version for the right syntax to use near '-sys-maint@localhost identified by 'j8KmNWOAiZuPbnfr'' at line 1
MariaDB [(none)]> grant all on *.* to 'debian-sys-maint'@localhost identified by 'j8KmNWOAiZuPbnfr';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit
Bye
db:~# /etc/init.d/mysql stop
Stopping MariaDB database server: mysqld.
db:~# tail /var/log/syslog
Nov 12 22:07:30 db mysqld: 101112 22:07:30 [Warning] Access denied for user 'debian-sys-maint'@'localhost' (using password: NO)
Nov 12 22:10:01 db mysqld: 101112 22:10:01 [Note] /usr/sbin/mysqld: Normal shutdown
Nov 12 22:10:01 db mysqld:
Nov 12 22:10:01 db mysqld: 101112 22:10:01 [Note] Event Scheduler: Purging the queue. 0 events
Nov 12 22:10:03 db mysqld: 101112 22:10:03 InnoDB: Starting shutdown...
Nov 12 22:10:07 db mysqld: 101112 22:10:07 InnoDB: Shutdown completed; log sequence number 45356
Nov 12 22:10:07 db mysqld: 101112 22:10:07 [Note] PrimeBase XT Engine shutdown...
Nov 12 22:10:07 db mysqld: 101112 22:10:07 [Note] /usr/sbin/mysqld: Shutdown complete
Nov 12 22:10:07 db mysqld:
Nov 12 22:10:07 db mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
db:~# /etc/init.d/mysql start
Starting MariaDB database server: mysqld . ..
Checking for corrupt, not cleanly closed and upgrade needing tables..
db:~# tail /var/log/syslog

==== Syslog output
Nov 12 22:10:51 db mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Nov 12 22:10:51 db mysqld: 101112 22:10:51 [Warning] options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set
Nov 12 22:10:51 db mysqld: 101112 22:10:51 [Note] PrimeBase XT (PBXT) Engine 1.0.11-7 Pre-GA loaded...
Nov 12 22:10:51 db mysqld: 101112 22:10:51 [Note] Paul McCullagh, PrimeBase Technologies GmbH, http://www.primebase.org
Nov 12 22:10:52 db mysqld: InnoDB: The InnoDB memory heap is disabled
Nov 12 22:10:52 db mysqld: InnoDB: Mutexes and rw_locks use GCC atomic builtins
Nov 12 22:10:52 db mysqld: InnoDB: Compressed tables use zlib 1.2.3
Nov 12 22:10:52 db mysqld: 101112 22:10:52 InnoDB: highest supported file format is Barracuda.
Nov 12 22:10:52 db mysqld: 101112 22:10:52 Percona XtraDB (http://www.percona.com) 1.0.12-12.1 started; log sequence number 45356
Nov 12 22:10:52 db mysqld: 101112 22:10:52 [Note] Event Scheduler: Loaded 0 events
Nov 12 22:10:52 db mysqld: 101112 22:10:52 [Note] /usr/sbin/mysqld: ready for connections.
Nov 12 22:10:52 db mysqld: Version: '5.2.3-MariaDB-mariadb87-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (MariaDB - http://mariadb.com/)
Nov 12 22:10:53 db /etc/mysql/debian-start[2871]: Upgrading MySQL tables if necessary.
Nov 12 22:10:54 db mysqld: 101112 22:10:54 [Warning] Statement may not be safe to log in statement format. Statement: ALTER TABLE general_log
Nov 12 22:10:54 db mysqld: MODIFY event_time TIMESTAMP NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY user_host MEDIUMTEXT NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY thread_id INTEGER NOT NULL,
Nov 12 22:10:54 db mysqld: MODIFY server_id ...

[Kristian Nielsen (knielsen)]

Thanks for the detailed report, I was able to repeat it. I will see if I can find a way to make it more robust.

[Kristian Nielsen (knielsen)]

Actually, the problem is not related to the many failed install attempts, it is much worse.
The script that is supposed to create the debian-sys-maint account has not been
updated to 5.2, which has a new column `auth_string` in the mysql.user table.
This means that any install of MariaDB 5.2 .debs will have the problem with missing
debian-sys-maint account! :-(
I'll get a fix cooked up.

Walter, thanks a million for bringing this very serious problem to our attention, as well
as for the excellent bug report!

[Kristian Nielsen (knielsen)]

I committed a fix for this in ourdelta-montyprogram-fixes (for failure to create debian-sys-maint account and default config causing warning about slow log options).

[Kristian Nielsen (knielsen)]

Summary of effect on users:

This affects new installs of .debs for MariaDB 5.2. Upgrading from earlier version is not affected, nor is 5.1 or installing from other source than .deb.

The symptom is that /etc/init.d/mysql stop does not work, nor does reload, status, etc.

We will upload new .deb packages shortly. Installing the new package should fix the problem. However, as shutdown of the old server is broken, it may be necessary to manually stop the old server with something like this:

    sudo kill -TERM $(pidof mysqld)

The new, fixed .deb packages have version string 5.2.3-mariadb90, while the old ones have version string 5.2.3-mariadb87

[Walter Heck (walterheck)]

Haven't verified but since we're at 5.2.4 now, I presume this is released?

[Arjen Lentz (arjen-lentz)]

just marking this as "invalid" in OurDelta project, since MariaDB 5.2 is built within the Maria project (its build scripts are just based on OurDelta's, but naturally evolve)

[Kristian Nielsen (knielsen)]

Yes, this is fixed in MariaDB 5.2.4.