Unattended-upgrade failed to restart mysql service (due to timeout?)

[Jeenson Aguilar Correa]

package mysql-server-8.0 8.0.25-0ubuntu0.20.04.1 failed to install/upgrade: installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1

ProblemType: Package
DistroRelease: Ubuntu 20.04
Package: mysql-server-8.0 8.0.25-0ubuntu0.20.04.1
ProcVersionSignature: Ubuntu 5.8.0-48.54~20.04.1-generic 5.8.18
Uname: Linux 5.8.0-48-generic x86_64
ApportVersion: 2.20.11-0ubuntu27.17
Architecture: amd64
CasperMD5CheckResult: skip
Date: Sat May 22 19:08:08 2021
ErrorMessage: installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
InstallationDate: Installed on 2020-05-08 (379 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Release amd64 (20200423)
Logs.var.log.daemon.log:

MySQLConf.etc.mysql.conf.d.mysql.cnf: [mysql]
MySQLConf.etc.mysql.conf.d.mysqldump.cnf:
 [mysqldump]
 quick
 quote-names
 max_allowed_packet = 16M
MySQLVarLibDirListing: ['the_spacebar', '#ib_16384_0.dblwr', 'auto.cnf', 'undo_001', 'public_key.pem', 'mysql_upgrade_info', 'ib_buffer_pool', 'client-key.pem', 'ib_logfile1', 'private_key.pem', 'ib_logfile0', 'mysql.ibd', 'ibdata1', 'sys', 'binlog.000024', 'ca.pem', 'ca-key.pem', 'undo_002', '#ib_16384_1.dblwr', 'client-cert.pem', 'server-cert.pem', 'vue_course', 'performance_schema', '#innodb_temp', 'debian-5.7.flag', 'binlog.000025', 'binlog.index', 'server-key.pem', 'binlog.000026', 'mysql']
ProcCmdline: BOOT_IMAGE=/boot/vmlinuz-5.8.0-48-generic root=UUID=ac22c095-f39d-4c27-bb4b-bc4e3cd1f9a9 ro quiet splash vt.handoff=7
Python3Details: /usr/bin/python3.8, Python 3.8.5, python3-minimal, 3.8.2-0ubuntu2
PythonDetails: N/A
RelatedPackageVersions:
 dpkg 1.19.7ubuntu3
 apt 2.0.5
SourcePackage: mysql-8.0
Title: package mysql-server-8.0 8.0.25-0ubuntu0.20.04.1 failed to install/upgrade: installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
UpgradeStatus: No upgrade log present (probably fresh install)

[Bryce Harrington (bryce)]

2021-05-23T00:04:43.500232Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.25-0ubuntu0.20.04.1).
2021-05-23T00:08:08.272503Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.25-0ubuntu0.20.04.1) (Ubuntu).

It looks like the upgrade process tried to stop the old service and start the new one, but complains that the old one didn't stop properly. Yet from the logs it appears it did stop, but took a few minutes to do it. Perhaps this was longer than the allowed timeout?

It's unclear why it took this long in this case; the log shows two shutdowns the previous day that took just a few seconds. Is it possible the system was under unusually high load? Or any other ideas why the database shutdown needed 4 minutes to complete?

[Klojum (klojum2001)]

I've had the same problem with just about every MySQL 8.0 upgrade on Ubuntu 20.04, and now again with 8.0.27. I had to kill both MySQL processes in a 2nd terminal screen to make the upgrade process continue in the 1st terminal screen where MySQL's upgrade was started. Simply stopping the MySQL server daemon beforehand is also not enough.

klojum@hp840:~$ ps -A | grep mysql
  17286 pts/1 00:00:00 mysql-server-8.
  17667 pts/1 00:00:01 mysqld
klojum@hp840:~$
klojum@hp840:~$ sudo kill -9 17286
[sudo] password for klojum:
klojum@hp840:~$ ps -A | grep mysql
  17286 pts/1 00:00:00 mysql-server-8. <defunct>
  17667 pts/1 00:00:01 mysqld
klojum@hp840:~$ sudo kill -9 17667
klojum@hp840:~$

And then Ubuntu's upgrade process continues.

[Sergio Durigan Junior (sergiodj)]

Thanks for providing further information, Klojum.

Since it seems like you can reproduce this problem, would you be able to do a few things in order to help us determine what might be going on?

Initially, I'm mostly interested in two things:

1) An strace log of the mysqld process that is being stopped.

2) A GDB backtrace of the same mysqld process.

For (1), it should be enough to just run strace on the server's PID. For (2), you will have to install the debuginfo files for the process before attaching to it. You can follow this guide:

https://wiki.ubuntu.com/Backtrace#Already_running_programs

Feel free to attach the logs to this bug.

Thank you in advance.

[Sergio Durigan Junior (sergiodj)]

FTR, this is likely a duplicate of https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1899248.

[Sergio Durigan Junior (sergiodj)]

@Klojum, another question: assuming you're able to reproduce this problem consistently, would you be willing to try a possible fix provided via a PPA? Thanks.