mysql-server postrm breaks apparmor profile for later versions on purge

[Alex Fomenko]

MySQL 5.1 and 5.5 must be rebuilt with the updated debhelper/apparmor and so cannot be verified until those fixes are complete.

Also for clarity sake, there has been a lot of shuffling of this bug around multiple packages for various reasons. This bug is fixed in quantal as part of the apparmor upload. All reverse deps should then be rebuilt for quantal. For SRU's, only known affected packages should be rebuilt, which thus far seems only to be mysql.

[Test case]
Apparmor:
1.Install dh-apparmor
2. Build a package which build-depends on it (such as mysql-5.5)
3. Extract debs which have dh_apparmor run on them into a dir in tmp with dpkg -e file.deb /tmp/foo
4. Verify that the postrm has conditional logic to not remove the files under /etc/apparmor.d/*/* if the main profile exists.

Debhelper.
Same test case as above, but with debhelper providing dh_apparmor

MySQL 5.1 and MySQL 5.5 (oneiric, precise, lucid)
1. install mysql-server
2. check /var/lib/dpkg/info/mysql-server-5.1.postrm for conditional logic (change 5.1 to 5.5 for precise)
3. for oneiric and lucid, upgrade to Ubuntu 12.04, verify that /etc/apparmor.d/local/usr.sbin.mysqld is still present.

[Regression Potential]
The logic is extremely straight forward, so regressions seem unlikely. Still, at worst, purging will leave behind some useless files in /etc/apparmor.d which should not be of much concern anyway.

== Original bug description ==

After upgrading from Kubuntu 11.10 to 12.04 beta I found that mysql isn't running after booting the system

In the syslog there is the following row
Apr 22 18:04:13 deathworld kernel: [ 15.848658] init: mysql pre-start process (1033) terminated with status 1

I've tried to execute /etc/init/mysql.conf step by step and found that execution
/lib/init/apparmor-profile-load usr.sbin.mysqld
shows the following error:
AppArmor parser error for /etc/apparmor.d/usr.sbin.mysqld in /etc/apparmor.d/usr.sbin.mysqld at line 44: Could not open 'local/usr.sbin.mysqld'
It's line:
#include <local/usr.sbin.mysqld>

After removing this line mysql can start without any problems

Description: Ubuntu 12.04 LTS
Release: 12.04

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Paco Soberón (unratito)]

I fixed it by creating a dummy /etc/apparmor.d/local/usr.sbin.mysqld file, rather than removing the offending line in /etc/apparmor.d/usr.sbin.mysqld.

I guess it could be a packaging problem, where the local file has been forgotten to be added.

[Alex Fomenko (falex007)]

I absolutely agree with you. My solution was made in a hurry

[Geoff (geoffch)]

I fixed the apparmor problem by copying /etc/apparmor.d/local/usr.sbin.mysqld from another machine that was not yet upgraded (this file is empty excepet for a comment). However, mysql doesn't start properly even so, and repeatedly respawns until init stops it:

May 2 16:23:02 rocky kernel: [ 2036.363661] init: mysql main process ended, respawning
May 2 16:23:02 rocky kernel: [ 2036.377177] type=1400 audit(1335997382.943:39): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=4578 comm="apparmor_parser"
May 2 16:23:05 rocky /etc/mysql/debian-start[4691]: Upgrading MySQL tables if necessary.
May 2 16:23:05 rocky /etc/mysql/debian-start[4694]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
May 2 16:23:05 rocky /etc/mysql/debian-start[4694]: Looking for 'mysql' as: /usr/bin/mysql
May 2 16:23:05 rocky /etc/mysql/debian-start[4694]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
May 2 16:23:05 rocky /etc/mysql/debian-start[4694]: This installation of MySQL is already upgraded to 5.5.22, use --force if you still need to run mysql_upgrade
May 2 16:23:05 rocky /etc/mysql/debian-start[4705]: Checking for insecure root accounts.
May 2 16:23:05 rocky /etc/mysql/debian-start[4710]: Triggering myisam-recover for all MyISAM tables
May 2 16:23:05 rocky kernel: [ 2038.631553] init: mysql main process (4585) terminated with status 1
May 2 16:23:05 rocky kernel: [ 2038.631632] init: mysql respawning too fast, stopped.

[Malcolm Scott (malcscott)]

As per bug #978995 this happens when an older mysql-server-* package, e.g. mysql-server-5.1, is purged post-upgrade.

The problem seems to be in dh_apparmor, which added the following to mysql-server-5.1.postrm:

# Automatically added by dh_apparmor
if [ "$1" = "purge" ]; then
    rm -f "/etc/apparmor.d/disable/usr.sbin.mysqld" || true
    rm -f "/etc/apparmor.d/force-complain/usr.sbin.mysqld" || true
    rm -f "/etc/apparmor.d/local/usr.sbin.mysqld" || true
    rmdir /etc/apparmor.d/local 2>/dev/null || true
fi
# End automatically added section

...which will remove config files which should not be removed as they continue to be used by mysql-server-5.5.

I'm not sure why dh_apparmor does not install /etc/apparmor.d/local/[profile] as a dpkg conffile; I suspect if it did this problem would not occur. Alternatively, the postrm script already works around this problem for dh_installinit, and this approach could also work for dh_apparmor.

[Jamie Strandboge (jdstrand)]

/etc/apparmor.d/local/usr.sbin.mysqld is intentionally not a conffile so that people can put their rules in there and not be prompted on upgrades. This seems like it should be fixed in mysql as part of the migration from 5.1 to 5.5.

[Jamie Strandboge (jdstrand)]

I believe the solution would be for the 5.1 packaging to not remove (or re-add) this file if /etc/apparmor.d/usr.sbin.mysqld exists.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Rob Johns (mstrtwelve)]

My fix was to add a space after the # in the statement #include <local/usr.sbin.mysqld> so that it became a comment. I don't like to delete things.

[Clint Byrum (clint-fewbar)]

I have tested installing mysql-server-5.1 in 11.10, and then doing a dist-upgrade to get to precise.

Ok so the problem is that mysql-server-5.1's postrm runs *after* mysql-server-5.5's postinst creates the file. This is because we only have a 'Breaks+Replaces' relationship between the two, so dpkg correctly:

* deconfigures 5.1
* installs 5.5
* configures 5.5
* removes 5.1

This really does need to be fixed in dh_apparmor. The postrm snippet should simply not rm the local/ file if the primary file is still in existence, since it is basically guaranteed to still depend on the local/ file.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Clint Byrum (clint-fewbar)]

Adding lucid task. This actually has to be fixed in lucid and oneiric so that mysql-5.1 -> 5.5 transitions go properly. Also adding mysql-5.1 since it will have to be rebuilt.

[Clint Byrum (clint-fewbar)]

dh_apparmor was in debhelper until precise, so oneiric/lucid apparmor is unaffected, but adding debhelper tasks instead

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.7.102-0ubuntu5

---------------
apparmor (2.7.102-0ubuntu5) quantal; urgency=low

  * debian/debhelper/postrm.apparmor: do not delete local files if main
    conffile still exists since it probably means it is owned by a
    new/different package. (LP: #986892)
 -- Clint Byrum <email address hidden> Mon, 11 Jun 2012 21:40:33 -0700

[Clint Byrum (clint-fewbar)]

dh_apparmor was added after lucid, so the code is directly in lucid's mysql-server-5.1.postrm

[Launchpad Janitor (janitor)]

This bug was fixed in the package mysql-5.5 - 5.5.25-0ubuntu1

---------------
mysql-5.5 (5.5.25-0ubuntu1) quantal; urgency=low

  * New upstream release (LP: #1011371, LP: #986892)
  * d/rules: change get-orig-source to pull from a working mirror.
  * d/control: Build with default compiler instead of gcc 4.5
 -- Clint Byrum <email address hidden> Mon, 11 Jun 2012 23:34:14 -0700

[Jamie Strandboge (jdstrand)]

I have verified that Clint's changes work-- eg I adjusted /var/lib/dpkg/info/tcpdump.postrm to have (ie, what dh_apparmor would add when tcpdump is built with new apparmor):
if [ "$1" = "purge" ] && ! [ -e "/etc/apparmor.d/usr.sbin.tcpdump" ]; then

and then I did:
$ sudo apt-get remove tcpdump # /etc/apparmor.d/local/usr.sbin.tcpdump still exists
$ sudo apt-get remove --purge tcpdump # /etc/apparmor.d/local/usr.sbin.tcpdump no longer exists

I will commit this to our bzr branch. Thanks Clint! :)

[Scott Kitterman (kitterman)]

Hello Alex, or anyone else affected,

Accepted into precise-proposed. The package will build now and be available in a few hours in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[MMlosh (mmlosh)]

I am not sure how would I reliably test this, sorry.

[Aymeric (mulx)]

@Scott

I'm unable to restore the environment in which I found that bug, looks like an upgrade of an other package changed something in apparmor that fix this bug.

I will try to upgrade (to precise) a backup of my system taken in December (version is natty) with and without -proposed.

Stay in touch for result.

Regards,
Aymeric.

[Clint Byrum (clint-fewbar)]

You do not need the originally affected system to verify this fix. There is a "test case" listed in the bug description which does not require a previously affected system, but does require one which you can upgrade to precise (A bare VM is useful here)

MySQL 5.1 and MySQL 5.5 (oneiric, precise, lucid)
1. install mysql-server
2. check /var/lib/dpkg/info/mysql-server-5.1.postrm for conditional logic (change 5.1 to 5.5 for precise)
3. for oneiric and lucid, upgrade to Ubuntu 12.04, verify that /etc/apparmor.d/local/usr.sbin.mysqld is still present.

[Thomas Tanghus (tanghus)]

@Clint: You suggested that the comment on http://tanghus.net/2012/03/yet-another-mysql-vs-apparmor-barf/#comment-298 is related, which seems to be the case. What I experienced wasn't a missing /etc/apparmor.d/local/usr.sbin.mysqld, but rather that the update would have changed the paths back to the not-working /var/run/* paths after I had manually corrected them to use /run prior to upgrading to Precise.
I didn't have an image earlier than 12.04, but tried to install mysql-server in a VM (it wasn't available in proposed anymore?). I'm not sure what conditional logic to look for in /var/lib/dpkg/info/mysql-server-5.5.postrm. /var/lib/dpkg/info/mysql-server-5.1.postrm does get removed on purge, but I assume that is intended behaviour?

/etc/apparmor.d/local/usr.sbin.mysqld is empty except for:

 # Site-specific additions and overrides for usr.sbin.mysqld.
 # For more details, please see /etc/apparmor.d/local/README.

/etc/apparmor.d/usr.sbin.mysqld otoh contains both:

  /var/run/mysqld/mysqld.pid w,
  /var/run/mysqld/mysqld.sock w,
  /run/mysqld/mysqld.pid w,
  /run/mysqld/mysqld.sock w,

But mysql runs fine:

Aug 14 14:34:30 NemID-VirtualBox kernel: [ 1530.635603] type=1400 audit(1344947670.136:27): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=6072 comm="apparmor_parser"
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6116]: Upgrading MySQL tables if necessary.
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6119]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6119]: Looking for 'mysql' as: /usr/bin/mysql
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6119]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6119]: This installation of MySQL is already upgraded to 5.5.24, use --force if you still need to run mysql_upgrade
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6130]: Checking for insecure root accounts.
Aug 14 14:34:32 NemID-VirtualBox /etc/mysql/debian-start[6135]: Triggering myisam-recover for all MyISAM tables

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 2.7.102-0ubuntu3.4

---------------
apparmor (2.7.102-0ubuntu3.4) precise-security; urgency=low

  * debian/debhelper/postrm.apparmor: do not delete local files if main
    conffile still exists since it probably means it is owned by a
    new/different package. (LP: #986892)
 -- Marc Deslauriers <email address hidden> Sat, 20 Oct 2012 16:55:18 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package mysql-dfsg-5.1 - 5.1.66-0ubuntu0.10.04.1

---------------
mysql-dfsg-5.1 (5.1.66-0ubuntu0.10.04.1) lucid-security; urgency=low

  * SECURITY UPDATE: Update to 5.1.66 to fix security issues (LP: #1068158)
    - http://www.oracle.com/technetwork/topics/security/cpuoct2012-1515893.html
  * debian/mysql-server-5.1.postrm: do not delete local files if main
    conffile still exists since it probably means it is owned by a
    new/different package. (LP: #986892)
 -- Marc Deslauriers <email address hidden> Fri, 19 Oct 2012 08:32:19 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package mysql-5.1 - 5.1.66-0ubuntu0.11.10.2

---------------
mysql-5.1 (5.1.66-0ubuntu0.11.10.2) oneiric-security; urgency=low

  * SECURITY UPDATE: Update to 5.1.66 to fix security issues (LP: #1068158)
    - http://www.oracle.com/technetwork/topics/security/cpuoct2012-1515893.html
  * debian/mysql-server-5.1.postrm: do not delete local files if main
    conffile still exists since it probably means it is owned by a
    new/different package. (LP: #986892)
 -- Marc Deslauriers <email address hidden> Mon, 22 Oct 2012 08:15:00 -0400

[dino99 (9d9)]

EOL reached now