mysql server crashes with port 3306 already in use

Bug #1921378 reported by Raghvendra Khatri
118
This bug affects 16 people
Affects Status Importance Assigned to Milestone
mysql-8.0 (Ubuntu)
Fix Released
High
Lena Voytek
Focal
In Progress
High
Lena Voytek
Impish
In Progress
High
Lena Voytek
Jammy
Fix Committed
High
Lena Voytek

Bug Description

[Impact]

When a user attepts to install mysql-server to their system, there is a chance that it will fail due to a race condition in the post-install script.
In this situation, the user will be provided with the error:

dpkg: error processing package mysql-server-8.0 (--configure):
 installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1

The only way to fix this is to uninstall then retry the install again.

The provided fix resolves the race condition by making sure mysql is shut down completely after setup before it boots back up again for use by the user.

[Test Plan]

Since this is a race condition, it can take a while to reproduce. However, using less cores and lower CPU speeds makes it appear more often. Testing with the following code many times will eventually show that, prior to the update, this issue appears. Meanwhile after the update it does not.

# lxc launch images:ubuntu/[kinetic, jammy, impish, focal] test-mysql
# lxc exec test-mysql bash
# apt update && apt dist-upgrade -y
# apt install mysql-server

[Where problems could occur]

Since this update has the postinst script enforce shutdown after configuration, mysql may be cut off abruptly, which could cause internal problems that it may need to address on the following startup.

[Other Info]

The fix was released to kinetic in 8.0.29-0ubuntu4

[Original Description]

mysql server not running

ProblemType: Package
DistroRelease: Ubuntu 20.04
Package: mysql-server-8.0 8.0.23-0ubuntu0.20.04.1
ProcVersionSignature: Ubuntu 5.8.0-48.54~20.04.1-generic 5.8.18
Uname: Linux 5.8.0-48-generic x86_64
ApportVersion: 2.20.11-0ubuntu27.16
Architecture: amd64
CasperMD5CheckResult: skip
Date: Thu Mar 25 15:33:48 2021
ErrorMessage: installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
InstallationDate: Installed on 2021-03-24 (0 days ago)
InstallationMedia: Ubuntu 20.04.2.0 LTS "Focal Fossa" - Release amd64 (20210209.1)
Logs.var.log.daemon.log:

MySQLConf.etc.mysql.conf.d.mysql.cnf: [mysql]
MySQLConf.etc.mysql.conf.d.mysqldump.cnf:
 [mysqldump]
 quick
 quote-names
 max_allowed_packet = 16M
MySQLVarLibDirListing: ['#innodb_temp', 'ca-key.pem', 'binlog.000008', 'client-key.pem', 'server-key.pem', 'sys', 'binlog.000007', 'ibtmp1', 'server-cert.pem', 'binlog.000004', 'client-cert.pem', 'undo_002', 'binlog.000013', 'mysql', 'performance_schema', 'ib_logfile0', 'auto.cnf', 'binlog.000005', 'binlog.000002', 'ib_logfile1', 'debian-5.7.flag', 'binlog.000001', 'private_key.pem', 'binlog.000015', 'binlog.000011', 'ibdata1', 'ca.pem', 'binlog.000009', 'binlog.index', 'public_key.pem', 'binlog.000003', 'ib_buffer_pool', 'binlog.000014', 'binlog.000016', '#ib_16384_0.dblwr', 'binlog.000006', 'mysql.ibd', '#ib_16384_1.dblwr', 'binlog.000012', 'raghvendra-HP-EliteBook-Folio-9470m.pid', 'undo_001', 'binlog.000010']
ProcCmdline: BOOT_IMAGE=/boot/vmlinuz-5.8.0-48-generic root=UUID=7d0f25d1-2e3e-4c9a-aabc-c337c6b1278b ro quiet splash vt.handoff=7
Python3Details: /usr/bin/python3.8, Python 3.8.5, python3-minimal, 3.8.2-0ubuntu2
PythonDetails: N/A
RelatedPackageVersions:
 dpkg 1.19.7ubuntu3
 apt 2.0.4
SourcePackage: mysql-8.0
Title: package mysql-server-8.0 8.0.23-0ubuntu0.20.04.1 failed to install/upgrade: installed mysql-server-8.0 package post-installation script subprocess returned error exit status 1
UpgradeStatus: No upgrade log present (probably fresh install)

Related branches

Revision history for this message
Raghvendra Khatri (raghvendrakhatri) wrote :
tags: removed: need-duplicate-check
Revision history for this message
Utkarsh Gupta (utkarsh) wrote :

Hello Raghvendra,

Thanks for taking out time to file a bug and making Ubuntu server better.

This bug is a duplicate of LP: #1899248 (cf: https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1899248) and thus I've marked the same. This is being worked out and people are aware of this already.

In case you think this isn't a duplicate or should you need any further assistance, please don't hesitate to reach out. Thanks, again.

Revision history for this message
Lena Voytek (lvoytek) wrote :

Hi,

After looking further into bug reports marked as duplicates of LP: #1899248, I noticed a separate error that is common between some of them. Although this is marked as a duplicate due to the error:

[ERROR] [MY-011300] [Server] Plugin mysqlx reported: 'Setup of socket: '/var/run/mysqld/mysqlx.sock' failed, can't create lock file /var/run/mysqld/mysqlx.sock.lock'

the report was more likely generated due to the following in the mysql error.log:

2021-03-25T10:57:58.560178Z 0 [ERROR] [MY-010262] [Server] Can't start server: Bind on TCP/IP port: Address already in use
2021-03-25T10:57:58.560457Z 0 [ERROR] [MY-010257] [Server] Do you already have another mysqld server running on port: 3306 ?

Therefore I will remove the duplicate status of this bug and use it for all bugs related to this specific issue.

Thanks

Changed in mysql-8.0 (Ubuntu):
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Lena Voytek (lvoytek)
tags: added: server-todo
Lena Voytek (lvoytek)
Changed in mysql-8.0 (Ubuntu):
status: Confirmed → In Progress
Lena Voytek (lvoytek)
summary: - package mysql-server-8.0 8.0.23-0ubuntu0.20.04.1 failed to
- install/upgrade: installed mysql-server-8.0 package post-installation
- script subprocess returned error exit status 1
+ mysql server crashes with port 3306 already in use
Revision history for this message
Utkarsh Gupta (utkarsh) wrote :

Hey, is there any update on this, Lena? :D

Revision history for this message
Lena Voytek (lvoytek) wrote :

Just started work on this again last week. I'm still working on reproducing the error consistently

Lena Voytek (lvoytek)
Changed in mysql-8.0 (Ubuntu Focal):
status: New → Confirmed
Changed in mysql-8.0 (Ubuntu Jammy):
status: New → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mysql-8.0 (Ubuntu Impish):
status: New → Confirmed
Lena Voytek (lvoytek)
Changed in mysql-8.0 (Ubuntu Focal):
assignee: nobody → Lena Voytek (lvoytek)
Changed in mysql-8.0 (Ubuntu Impish):
assignee: nobody → Lena Voytek (lvoytek)
Changed in mysql-8.0 (Ubuntu Jammy):
assignee: nobody → Lena Voytek (lvoytek)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mysql-8.0 - 8.0.29-0ubuntu4

---------------
mysql-8.0 (8.0.29-0ubuntu4) kinetic; urgency=medium

  * d/mysql-server-8.0.postinst: Confirm mysqld shuts down with stop_server after
    initialization to avoid overlapping use of port 3306 (LP: #1921378)

 -- Lena Voytek <email address hidden> Wed, 15 Jun 2022 14:27:47 -0700

Changed in mysql-8.0 (Ubuntu):
status: In Progress → Fix Released
Lena Voytek (lvoytek)
Changed in mysql-8.0 (Ubuntu Focal):
status: Confirmed → In Progress
Changed in mysql-8.0 (Ubuntu Impish):
status: Confirmed → In Progress
Changed in mysql-8.0 (Ubuntu Jammy):
status: Confirmed → In Progress
Changed in mysql-8.0 (Ubuntu Focal):
importance: Undecided → High
Changed in mysql-8.0 (Ubuntu Impish):
importance: Undecided → High
Changed in mysql-8.0 (Ubuntu Jammy):
importance: Undecided → High
Lena Voytek (lvoytek)
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Raghvendra, or anyone else affected,

Accepted mysql-8.0 into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mysql-8.0/8.0.29-0ubuntu0.22.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in mysql-8.0 (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-jammy
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (mysql-8.0/8.0.29-0ubuntu0.22.04.3)

All autopkgtests for the newly accepted mysql-8.0 (8.0.29-0ubuntu0.22.04.3) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

exim4/4.95-4ubuntu2.1 (ppc64el)
ruby-mysql2/0.5.3-3ubuntu4 (armhf, amd64, s390x, arm64, ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#mysql-8.0

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

To post a comment you must log in.