MySQL server fails to upgrade in Gutsy - debian-sys-maint user permission denied

[Emil Oppeln-Bronikowski]

After last beta release of Gutsy Gibbon, MySQL server fails to upgrade.

Here's the aptitiude log of events:

Preparing to replace mysql-server-5.0 5.0.45-1ubuntu2 (using .../mysql-server-5.0_5.0.45-1ubuntu3_i386.deb) ...
 * Stopping MySQL database server mysqld [fail]
invoke-rc.d: initscript mysql, action "stop" failed.
dpkg: warning - old pre-removal script returned error exit status 1
dpkg - trying script from the new package instead ...
 * Stopping MySQL database server mysqld [fail]
invoke-rc.d: initscript mysql, action "stop" failed.
dpkg: error processing /var/cache/apt/archives/mysql-server-5.0_5.0.45-1ubuntu3_i386.deb (--unpack):
 subprocess new pre-removal script returned error exit status 1
 * Stopping MySQL database server mysqld [fail]
invoke-rc.d: initscript mysql, action "stop" failed.
 * Starting MySQL database server mysqld [ OK ]
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using password: YES)'

This is stock installation, I didn't change configuration files, so wrong hash for debian-sys-maint is out of the question.

Here's a current package informations:

Package: mysql-server
Priority: optional
Section: misc
Installed-Size: 84
Maintainer: Ubuntu Core Developers <email address hidden>
Original-Maintainer: Debian MySQL Maintainers <email address hidden>
Architecture: all
Source: mysql-dfsg-5.0
Version: 5.0.45-1ubuntu3
Depends: mysql-server-5.0
Filename: pool/main/m/mysql-dfsg-5.0/mysql-server_5.0.45-1ubuntu3_all.deb
Size: 49768
MD5sum: 3771e04e6abd940a037df89e4addfa35
SHA1: 51a01df0a64d4d629a3bdbe6172ba51e76172870
SHA256: a620d77257b6ec22cae25e33eb713be0b3eb00aa330340506b6a70e51f0ceb3d

[chastell (chastell)]

I can confirm this on Feisty with mysql-server upgrade attempt 5.0.38-0ubuntu1 → 5.0.38-0ubuntu1.1

[Ante Karamatić (ivoks)]

Did you change permissions for users in mysql.user table?

[Jamie Strandboge (jdstrand)]

Thank you for your bug report and helping make Ubuntu better. At this time we do not have enough information to diagnose the bug. Please check your logs in /var/log and attach the log messages pertaining to this issue.

It is also possible that the Feisty issue is a different issue, so it would be great to get these logs as well.

[Adam Sommer (asommer)]

For what it's worth I was able to update MySQL on Feisty just fine:

Setting up mysql-client-5.0 (5.0.38-0ubuntu1.1) ...
Setting up mysql-server-5.0 (5.0.38-0ubuntu1.1) ...
Installing new version of config file /etc/init.d/mysql ...
 * Stopping MySQL database server mysqld [ OK ]
 * Starting MySQL database server mysqld [ OK ]
 * Checking for corrupt, not cleanly closed and upgrade needing tables.

Setting up mysql-server (5.0.38-0ubuntu1.1) ...
Setting up mysql-client (5.0.38-0ubuntu1.1) ...

I used "apt-get update; apt-get upgrade" though not aptitude.

[Adam Sommer (asommer)]

I was also able to update on Gutsy without issue:

Preparing to replace mysql-server-5.0 5.0.45-1ubuntu2 (using .../mysql-server-5.0_5.0.45-1ubuntu3_i386.deb) ...

Setting up mysql-server-5.0 (5.0.45-1ubuntu3) ...
 * Stopping MySQL database server mysqld [ OK ]
 * Starting MySQL database server mysqld [ OK ]
 * Checking for corrupt, not cleanly closed and upgrade needing tables.

Again, I used "sudo apt-get update; sudo apt-get upgrade" and not aptitude so I guess it could be an aptitude issue?

[chastell (chastell)]

I (the Feisty case) didn’t touch mysql.user at all, and all previous MySQL security upgrades went flawlessly. Also, my Gutsy devel box (upgraded from Feisty a couple of weeks ago) upgraded just fine this time.

Note: On my Feisty server, debian-sys-maint seems to have ALL PRIVILEGES, while on my Gutsy devel laptop it seems to have SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, FILE, REFERENCES, INDEX, ALTER, SHOW DATABASES, SUPER, CREATE TEMPORARY TABLES, LOCK TABLES, REPLICATION SLAVE, REPLICATION CLIENT, EXECUTE.

On the Feisty machine, all my /var/log/mysql.{log,err}* files are empty (again, permission issue?). :|

[Emil Oppeln-Bronikowski (opi)]

Ho hum! It seems that after all my mysql.user Password field was different from /etc/mysql/debian.cnf. I've updated password string.

mysql> SELECT Host,User,Password FROM user WHERE User = 'debian-sys-maint';
+-----------+------------------+------------------+
| Host | User | Password |
+-----------+------------------+------------------+
| localhost | debian-sys-maint | 69006d5d4d87514c |
+-----------+------------------+------------------+
1 row in set (0.00 sec)

root@heroina:~# cat /etc/mysql/debian.cnf | grep password
password = 69006d5d4d87514c
password = 69006d5d4d87514c

And now... MySQL won't even start.

root@heroina:~# /etc/init.d/mysql start
 * Starting MySQL database server mysqld [ OK ]
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using password: YES)'

I wonder if that's somehow related to Host field. Maybe I'll change its value to '%'. About logs, same as in Shot comment before, /var/log/mysql.err and .log are empty. Binlogs are also quiet.

[Mathias Gug (mathiaz)]

@Emil: Could you check that the user debian-sys-maint existed before the upgrade ? Can you try to connect to the database with its credentials (which are located in /etc/mysql/debian.cnf) ?

@Shot: do you see the same error message (debian-sys-maint permission denied) ? If could you try the same test specified above ?

[Mathias Gug (mathiaz)]

Emil: Which command did you use to set debian-sys-maint password ? Can you connect with the mysql client ?

[Emil Oppeln-Bronikowski (opi)]

OK, neither OLD_PASSWORD nor PASSWORD didn't produce good results. My mysql.user for debian-sys-maint looks not like this:

mysql> select User, Password, Host from user where User like 'debian%';
+------------------+-------------------------------------------+------+
| User | Password | Host |
+------------------+-------------------------------------------+------+
| debian-sys-maint | *DCE05059E58FA45F45A40420DF3F02D16D103E73 | % |
+------------------+-------------------------------------------+------+
1 row in set (0.00 sec)

It still fails to start, but, if I run mysqld by hand, I can log in without problems using same password I did hash from.

emil@heroina:/var/www/swatka$ mysql -p -u debian-sys-maint
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 5.0.45-Debian_1ubuntu2-log Debian etch distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

I'm 99% sure now that there's a problem with upgrade scripts in package.

[Emil Oppeln-Bronikowski (opi)]

OK, I'm fscking dumb! ;-)

Good think someone been watching behind my shoulder.

The 'password' var in debian.cfg SHOULD NOT CONSIST HASH :D

*HEADSHOT*

After I changed that to plain text password that matches mysql.user.debian-sys-maint it works again. Lets see if it will process dist-upgrade.

OK, it failed to use debian-sys-maint again, but it upgraded. Still, I think there's bug somewhere. Maybe Shot will be able to pin it down.

[Mathias Gug (mathiaz)]

What do you mean by it failed to use debian-sys-maint again ?

[Ante Karamatić (ivoks)]

I can just confirm that update went without problems on >20 servers (mostly Dapper, couple of Feistys and 3 Hardys). I always upgrade with apt-get.

[Soren Hansen (soren)]

On Tue, Oct 16, 2007 at 02:34:12PM -0000, Emil Oppeln-Bronikowski wrote:
>
> mysql> SELECT Host,User,Password FROM user WHERE User =
> 'debian-sys-maint';
> +-----------+------------------+------------------+
> | Host | User | Password |
> +-----------+------------------+------------------+
> | localhost | debian-sys-maint | 69006d5d4d87514c |
> +-----------+------------------+------------------+
> 1 row in set (0.00 sec)
>
> root@heroina:~# cat /etc/mysql/debian.cnf | grep password
> password = 69006d5d4d87514c
> password = 69006d5d4d87514c

Er, the password in the user table is hashed, so it shouldn't match
what's in the .conf file.

--
Soren Hansen
Ubuntu Server Team
http://www.ubuntu.com/

[Soren Hansen (soren)]

On Tue, Oct 16, 2007 at 04:19:15PM -0000, Ante Karamatić wrote:
> (mostly Dapper, couple of Feistys and 3 Hardys). I always upgrade with
> apt-get.

Hardys? Wow, you're more 1337 than I thought! :)

--
Soren Hansen
Ubuntu Server Team
http://www.ubuntu.com/

[Emil Oppeln-Bronikowski (opi)]

@Soren: yes, I stated I figured it out on my own. I can not reproduce it anymore and I think, if that's OK with Shot who had similar problem, we can close it.

[chastell (chastell)]

After investigating this thoroughly, it turned out debian-sys-maint’s password differed between /etc/mysql/debian.cnt and the database. After a proper SET PASSWORD FOR 'debian-sys-maint'@'localhost' everything seems ok (and the upgrade ran smoothly), so feel free to close this bug.