mysql-server-5.7 5.7.25 upgrade did not complete properly

[Pepe Kubon]

Ubuntu 18.04.1 LTS
set to run unattended upgrades
log shows there was an issue during the mysql-server 5.7.25 upgrade

mysql --version shows
mysql Ver 14.14 Distrib 5.7.25, for Linux (x86_64) using EditLine wrapper

sudo service mysql restart
seems to work

sudo dpkg --configure -a says
"This installation of MySQL is already upgraded to 5.7.25, use --force if you still need to run mysql_upgrade"
but then it hangs

ProblemType: Package
DistroRelease: Ubuntu 18.04
Package: mysql-server-5.7 5.7.25-0ubuntu0.18.04.2
ProcVersionSignature: Ubuntu 4.15.0-43.46-generic 4.15.18
Uname: Linux 4.15.0-43-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.5
AptOrdering: NULL: ConfigurePending
Architecture: amd64
Date: Fri Feb 22 17:02:20 2019
ErrorMessage: installed mysql-server-5.7 package post-installation script subprocess returned error exit status 1
InstallationDate: Installed on 2018-11-25 (89 days ago)
InstallationMedia: Ubuntu 18.04.1 LTS "Bionic Beaver" - Release amd64 (20180725)
Logs.var.log.daemon.log:

MySQLConf.etc.mysql.conf.d.mysql.cnf: [mysql]
MySQLConf.etc.mysql.conf.d.mysqldump.cnf:
 [mysqldump]
 quick
 quote-names
 max_allowed_packet = 16M
MySQLConf.etc.mysql.mysql.conf.d.mysqld_safe_syslog.cnf:
 [mysqld_safe]
 syslog
MySQLVarLibDirListing: ['ib_buffer_pool', 'ibtmp1', 'auto.cnf', 'debian-5.7.flag', 'performance_schema', 'live_lpvm', 'ib_logfile0', 'mysql', 'ibdata1', 'sys', 'test_lpvm', 'ib_logfile1', 'mysql_upgrade_info', 'dev_lpvm']
ProcCmdline: BOOT_IMAGE=/vmlinuz-4.15.0-43-generic root=UUID=06d7f4a2-b2be-4d54-b648-df3576e8a2bf ro quiet splash vt.handoff=1
Python3Details: /usr/bin/python3.6, Python 3.6.7, python3-minimal, 3.6.7-1~18.04
PythonDetails: /usr/bin/python2.7, Python 2.7.15rc1, python-minimal, 2.7.15~rc1-1
RelatedPackageVersions:
 dpkg 1.19.0.5ubuntu2.1
 apt 1.6.6ubuntu0.1
SourcePackage: mysql-5.7
Title: package mysql-server-5.7 5.7.25-0ubuntu0.18.04.2 failed to install/upgrade: installed mysql-server-5.7 package post-installation script subprocess returned error exit status 1
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.mysql.mysql.conf.d.mysqld.cnf: [modified]
mtime.conffile..etc.mysql.mysql.conf.d.mysqld.cnf: 2018-12-03T19:08:33.074857

[Pepe Kubon (chateau-pp)]

Stuck on this - tried a bunch of things up to purging mysql-server and mysql-server-5.7 packages. Reinstalling proceeded fine until the same blocking message:

Checking if update is needed.
This installation of MySQL is already upgraded to 5.7.25, use --force if you still need to run mysql_upgrade

At this point the process hangs in the terminal until I stop it manually

dpkg -l | grep mysql-server-5.7 shows "iF" for the status flags

So now I'm really messed up because the problem blocks the reinstall of mysql-server. Any suggestions as to how to get out of this mess would be really appreciated.

-Pp

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andreas Hasenack (ahasenack)]

I'm seeing errors like these in your logs:
2019-02-22T22:01:48.488648Z 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-02-22T22:01:48.488703Z 0 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.

Maybe you have a stale mysql running while trying to install it again? I suggest to carefully inspect the list of running processes before trying again.

[Pepe Kubon (chateau-pp)]

Thanks Andreas - true but I think this was a symptom rather than a cause. That lock issue was happening after I ran

sudo dpkg --configure -a

This kept hanging with the message
"This installation of MySQL is already upgraded to 5.7.25, use --force if you still need to run mysql_upgrade"

I believe the error report got generated after one of these.

Trying to get this resolved, I'd restarted several times but kept getting into the same locking issue as long as I tried to make the mysql-server upgrade clear its installation state.

[Andreas Hasenack (ahasenack)]

I'm afraid this is leaving the realm of a bug report and entering one of support.

The upgrade path in the code can be seen in /var/lib/dpkg/info/mysql-server-5.7.postinst:

    start_server "$mysql_statedir" "$tmpdir" "skip_grant" || result=$?
    # If the server fails to start, then skip the various client operations
    if [ $result -ne 0 ]; then
      echo "Warning: Unable to start the server. Please restart MySQL and run mysql_upgrade to ensure the database is ready for use." >&2
    else
      # mysql_upgrade returns exit status 2 if the database is already upgraded
      # (LP: #1566406) so ignore its exit status if it is 2.
      result=0
      mysql_upgrade --no-defaults --socket="$tmpdir/mysqld.sock" || result=$?
      if [ $result -ne 0 -a $result -ne 2 ]; then
        echo "mysql_upgrade failed with exit status $result" >&2
        stop_server "$tmpdir"
        rm -rf "$tmpdir"
        exit 1
      fi
      # Stop the server
      stop_server "$tmpdir"
    fi

I suppose you could try to replicate that mysql_upgrade command to see if you can clear the upgrade status, or maybe add the suggested --force parameter and try dpkg --configure -a again. And make sure there is no running mysql process.

That being said, if purging and starting over is an option for you, that should also work. Just make sure you purge all packages (and not just remove).

[Launchpad Janitor (janitor)]

[Expired for mysql-5.7 (Ubuntu) because there has been no activity for 60 days.]

[Burro-net (n-postmaster)]

It looks like it has something to do with having no password for the root user like on my dev box.

I have the same

        [ERROR] 1396 Operation ALTER USER failed for 'root'@'localhost'

in my error log and after every try a stale *.tmp file in /var/lib/mysql-files.

The "ALTER user" command comes from the file
 "/var/lib/dpkg/info/mysql-server-5.7.postinst"
You read in line 368:
"If there is no root password set we enable the auth_socket plugin for the root user."
This seems fails for some SSL related problems.

I could solve the block with these steps:

1. delete stale tmp files in /var/lib/mysql-files
2. Comment the line 370 which is :
echo "ALTER USER 'root'@'localhost' IDENTIFIED WITH 'auth_socket';" >> "$initfile"

[Ravindra Gullapalli (sharewithravig)]

I could not delete the tmp files in /var/lib/mysql-files

However, I commented line # 370 in "/var/lib/dpkg/info/mysql-server-5.7.postinst" and thereafter the installation went through.

[manu (manu.pintor)]

Hi, still affecting MySQL 5.7.34...

So, it looks like you need to check var/lib/dpkg/info/mysql-server-5.7.postinst every time MySQL has been updated, in case this file has been replaced by a new one, with the line 370 uncommented...

This is still an issue, although this bug has been closed "because of no activity"...

Maybe replacing MySQL with MariaDB would solve this?

[Lucas Kanashiro (lucaskanashiro)]

Thank you for taking the time to file a bug report.

I created a Bionic container to test the upgrade of mysql version 5.7.21-1ubuntu1 to 5.7.34-0ubuntu0.18.04.1 and it worked as expected:

root@mysql-test:~# apt list --upgradable
Listing... Done
mysql-server-5.7/bionic-updates,bionic-security 5.7.34-0ubuntu0.18.04.1 amd64 [upgradable from: 5.7.21-1ubuntu1]
mysql-server-core-5.7/bionic-updates,bionic-security 5.7.34-0ubuntu0.18.04.1 amd64 [upgradable from: 5.7.21-1ubuntu1]
root@mysql-test:~# apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following packages will be upgraded:
  mysql-server-5.7 mysql-server-core-5.7
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.3 MB of archives.
After this operation, 2650 kB disk space will be freed.
Do you want to continue? [Y/n]
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 mysql-server-5.7 amd64 5.7.34-0ubuntu0.18.04.1 [2905 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 mysql-server-core-5.7 amd64 5.7.34-0ubuntu0.18.04.1 [7418 kB]
Fetched 10.3 MB in 3s (3986 kB/s)
Preconfiguring packages ...
(Reading database ... 29454 files and directories currently installed.)
Preparing to unpack .../mysql-server-5.7_5.7.34-0ubuntu0.18.04.1_amd64.deb ...
Unpacking mysql-server-5.7 (5.7.34-0ubuntu0.18.04.1) over (5.7.21-1ubuntu1) ...
Preparing to unpack .../mysql-server-core-5.7_5.7.34-0ubuntu0.18.04.1_amd64.deb ...
Unpacking mysql-server-core-5.7 (5.7.34-0ubuntu0.18.04.1) over (5.7.21-1ubuntu1) ...
Setting up mysql-server-core-5.7 (5.7.34-0ubuntu0.18.04.1) ...
Setting up mysql-server-5.7 (5.7.34-0ubuntu0.18.04.1) ...
Installing new version of config file /etc/apparmor.d/usr.sbin.mysqld ...
Checking if update is needed.
Checking server version.
Running queries to upgrade MySQL server.
Checking system database.
mysql.columns_priv OK
mysql.db OK
mysql.engine_cost OK
mysql.event OK
mysql.func OK
mysql.general_log OK
mysql.gtid_executed OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.innodb_index_stats OK
mysql.innodb_table_stats OK
mysql.ndb_binlog_index OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.server_cost OK
mysql.servers OK
mysql.slave_master_info O...

[Launchpad Janitor (janitor)]

[Expired for mysql-5.7 (Ubuntu) because there has been no activity for 60 days.]