Upgrade fails when /var/lib/mysql is a soft link

Bug #1474212 reported by itsadok on 2015-07-14
96
This bug affects 20 people
Affects Status Importance Assigned to Milestone
mariadb-10.0 (Debian)
Fix Released
Unknown
mariadb-10.0 (Ubuntu)
Medium
Unassigned
Xenial
Undecided
Unassigned
mysql-5.7 (Ubuntu)
Medium
Lars Tangvald
Xenial
Medium
Lars Tangvald

Bug Description

Upgrade fails if datadir is a symlink and the server has previously been upgraded with datadir as a symlink

[Impact]
When upgrading, postinst makes a copy (with -d) of the datadir link to ensure it's preserved. This copy operation can fail in some situations if the destination already exists (because the package was previously upgraded with a similar situation), causing d/preinst to fail every time the user tries to upgrade the package unless they manually remove the old link every time.

The fix adds a -T to the copy operation, treating the destination as a regular file to be overwritten

[Test case]
* Create a /var/lib/mysql that is a symlink to some other location
* Install an older version of mysql-server
* Upgrade to a newer version of mysql-server

[Regression potential]
This change has been in Yakkety for some time (5.7.15-0ubuntu2).

The change only affects user with datadirs that are symlinks, and only a file copy operation for a file that's generally not used (it's backed up just in case)

[Original Description]
Not sure if it's relevant, by my mariadb data dir is on a secondary hard drive. /var/lib/mysql-upgrade/DATADIR.link points to the right path.

This is the terminal output for apt-get dist-upgrade:

Selecting previously unselected package mariadb-server-10.0.
(Reading database ... 274672 files and directories currently installed.)
Preparing to unpack .../mariadb-server-10.0_10.0.20-0ubuntu0.15.04.1_amd64.deb ...
cp: cannot overwrite directory ‘/var/lib/mysql-upgrade/DATADIR.link/mysql’ with non-directory
dpkg: error processing archive /var/cache/apt/archives/mariadb-server-10.0_10.0.20-0ubuntu0.15.04.1_amd64.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
+ . /usr/share/debconf/confmodule
++ '[' '!' '' ']'
++ PERL_DL_NONLAZY=1
++ export PERL_DL_NONLAZY
++ '[' '' ']'
++ exec /usr/share/debconf/frontend /var/lib/dpkg/info/mariadb-server-10.0.postinst abort-upgrade 10.0.20-0ubuntu0.15.04.1
+ . /usr/share/debconf/confmodule
++ '[' '!' 1 ']'
++ '[' -z '' ']'
++ exec
++ '[' '' ']'
++ exec
++ DEBCONF_REDIR=1
++ export DEBCONF_REDIR
+ '[' -n '' ']'
+ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin:/usr/sbin:/bin:/usr/bin
+ ERR_LOGGER='logger -p daemon.err -t mysqld_safe -i'
+ set -o pipefail
+ MYSQL_BOOTSTRAP='/usr/sbin/mysqld --bootstrap --user=mysql --skip-grant-tables --default-storage-engine=myisam --plugin-load-add=auth_socket'
+ set +e
+ invoke stop
+ '[' -x /usr/sbin/invoke-rc.d ']'
+ invoke-rc.d mysql stop
+ set -e
+ case "$1" in
+ '[' abort-upgrade = configure ']'
+ db_stop
+ echo STOP
+ '[' -x /etc/init.d/mysql ']'
+ update-rc.d mysql defaults 19 21
+ '[' -x /etc/init.d/mysql ']'
+ echo 'Invoking mysql start'
+ logger -p daemon.err -t mysqld_safe -i
+ invoke-rc.d mysql start
+ exit 0
Preparing to unpack .../mariadb-server_10.0.20-0ubuntu0.15.04.1_all.deb ...
Unpacking mariadb-server (10.0.20-0ubuntu0.15.04.1) over (10.0.17-0ubuntu1) ...
Errors were encountered while processing:
 /var/cache/apt/archives/mariadb-server-10.0_10.0.20-0ubuntu0.15.04.1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

ProblemType: Package
DistroRelease: Ubuntu 15.04
Package: mariadb-server-10.0 10.0.17-0ubuntu1
ProcVersionSignature: Ubuntu 3.19.0-22.22-generic 3.19.8-ckt1
Uname: Linux 3.19.0-22-generic x86_64
ApportVersion: 2.17.2-0ubuntu1.1
Architecture: amd64
Date: Tue Jul 14 08:07:15 2015
DpkgTerminalLog:
 Preparing to unpack .../mariadb-server-10.0_10.0.20-0ubuntu0.15.04.1_amd64.deb ...
 cp: cannot overwrite directory ‘/var/lib/mysql-upgrade/DATADIR.link/mysql’ with non-directory
 dpkg: error processing archive /var/cache/apt/archives/mariadb-server-10.0_10.0.20-0ubuntu0.15.04.1_amd64.deb (--unpack):
  subprocess new pre-installation script returned error exit status 1
DuplicateSignature: package:mariadb-server-10.0:10.0.17-0ubuntu1:subprocess new pre-installation script returned error exit status 1
ErrorMessage: subprocess new pre-installation script returned error exit status 1
InstallationDate: Installed on 2015-01-15 (180 days ago)
InstallationMedia: Ubuntu 14.10 "Utopic Unicorn" - Release amd64 (20141022.1)
RelatedPackageVersions:
 dpkg 1.17.25ubuntu1
 apt 1.0.9.7ubuntu4
SourcePackage: mariadb-10.0
Title: package mariadb-server-10.0 10.0.17-0ubuntu1 failed to install/upgrade: subprocess new pre-installation script returned error exit status 1
UpgradeStatus: Upgraded to vivid on 2015-05-27 (47 days ago)

CVE References

itsadok (itsadok) wrote :
tags: removed: need-duplicate-check
itsadok (itsadok) wrote :

The problem was an existing /var/lib/mysql-upgrade/DATADIR.link file, which caused the soft link backup mechanism to fail.

This can be fixed manually be deleting /var/lib/mysql-upgrade/DATADIR.link and running apt-get -f install.

To fix the preinst script, just change the line

cp -d "$checkdir" "$UPGRADEDIR/$dir.link"

to

cp -dT "$checkdir" "$UPGRADEDIR/$dir.link"

Otto Kekäläinen (otto) on 2015-07-23
Changed in mariadb-10.0 (Ubuntu):
status: New → Confirmed
Otto Kekäläinen (otto) on 2015-07-23
summary: - package mariadb-server-10.0 10.0.17-0ubuntu1 failed to install/upgrade:
- subprocess new pre-installation script returned error exit status 1
+ Upgrade fails when /var/lib/mysql is a soft link
Changed in mariadb-10.0 (Debian):
status: Unknown → New
Changed in mariadb-10.0 (Debian):
status: New → Fix Released
Changed in mariadb-10.0 (Ubuntu):
importance: Undecided → Medium
Otto Kekäläinen (otto) wrote :

Marking 'fix committed' as Ubuntu will soon sync latest version from Debian repos.

Changed in mariadb-10.0 (Ubuntu):
status: Confirmed → Fix Committed

same bug when upgrading to 16.04 LTS with mysql

installation report:

% sudo apt-get install mysql-server
Lecture des listes de paquets... Fait
Construction de l'arbre des dépendances
Lecture des informations d'état... Fait
The following additional packages will be installed:
  mysql-server-5.7
Paquets suggérés :
  mailx tinyca
Les NOUVEAUX paquets suivants seront installés :
  mysql-server mysql-server-5.7
0 mis à jour, 2 nouvellement installés, 0 à enlever et 0 non mis à jour.
Il est nécessaire de prendre 2 613 ko dans les archives.
Après cette opération, 48,6 Mo d'espace disque supplémentaires seront utilisés.
Souhaitez-vous continuer ? [O/n]
Réception de:1 http://fr.archive.ubuntu.com/ubuntu xenial/main amd64 mysql-server-5.7 amd64 5.7.11-0ubuntu6 [2 603 kB]
Réception de:2 http://fr.archive.ubuntu.com/ubuntu xenial/main amd64 mysql-server all 5.7.11-0ubuntu6 [10,1 kB]
2 613 ko réceptionnés en 1s (2 586 ko/s)
Préconfiguration des paquets...
(Lecture de la base de données... 251670 fichiers et répertoires déjà installés.)
Préparation du dépaquetage de .../mysql-server-5.7_5.7.11-0ubuntu6_amd64.deb ...
cp: impossible d'écraser le répertoire '/var/lib/mysql-upgrade/DATADIR.link/mysql' par un non répertoire
dpkg: erreur de traitement de l'archive /var/cache/apt/archives/mysql-server-5.7_5.7.11-0ubuntu6_amd64.deb (--unpack) :
 le sous-processus nouveau script pre-installation a retourné une erreur de sortie d'état 1
Sélection du paquet mysql-server précédemment désélectionné.
Préparation du dépaquetage de .../mysql-server_5.7.11-0ubuntu6_all.deb ...
Dépaquetage de mysql-server (5.7.11-0ubuntu6) ...
Des erreurs ont été rencontrées pendant l'exécution :
 /var/cache/apt/archives/mysql-server-5.7_5.7.11-0ubuntu6_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

% ls -ld /var/lib/mysql*
lrwxrwxrwx 1 mysql root 21 mars 16 17:16 /var/lib/mysql -> /home/Database/Mysql/
drwxr-xr-x 2 root root 4,0K avril 22 07:08 /var/lib/mysql-upgrade

Robie Basak (racb) on 2016-04-25
Changed in mysql-5.7 (Ubuntu):
importance: Undecided → Medium
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mysql-5.7 (Ubuntu):
status: New → Confirmed
Robie Basak (racb) on 2016-04-29
Changed in mysql-5.7 (Ubuntu):
milestone: none → ubuntu-16.06
pdf (pdffs) wrote :

@racb would it be possible to get an SRU for this into 16.04 since the change is purely packaging-related, and it would be nice if the LTS was not broken in this way in perpetuity?

Robie Basak (racb) wrote :

@pdf when we have a fix available, and depending on the invasiveness of the required fix, I would like to push it back to 16.04, yes. I'm focused on some other MySQL bugs that need fixing in 16.04 first though.

ironstorm (ironstorm-gmail) wrote :

@racb, what is wrong with the fix itsadok described on 2015-07-20?

I've manually unpacked and applied that fix to workaround this issue a few times (every time there's an updated package).

I don't understand why it takes so long to get things like this fixed when there's solution posted at the top of the ticket.

Let's apply the "T" to the package's preinst and be done with it already. :P

asavah (irherder) wrote :

@racb Is there any possibility that you apply the fix described at https://bugs.launchpad.net/ubuntu/+source/mariadb-10.0/+bug/1474212/comments/2 to mysql-server package?
It has been over a YEAR since this bug was open, the fix is known and works properly,
it just ONE letter in the script, it's not an upstream issue, it's just debian/ubuntu package bug.
I'm sick of fixing and manually repacking the deb every time an update arrives ...

Please get you lazy ass to fix this asap.

Richard (ismail-a) wrote :

I had this on 160724 and 160623 on update of mysql-common

IMPACT:
hangs unattended apt-get upgrade
blocks package operations

GETAROUND:
rm /var/lib/mysql-upgrade/DATADIR.link
apt-get --fix-broken install

Richard (ismail-a) wrote :

and SYMPTOM:
$ apt-get --yes dist-upgrade

Setting up mysql-common (5.7.13-0ubuntu0.16.04.2) ...

cp: cannot overwrite directory '/var/lib/mysql-upgrade/DATADIR.link/mysql' with non-directory

E: Sub-process /usr/bin/dpkg returned an error code (1)

Richard (ismail-a) wrote :

and CAUSE:
mysqld --no-defaults --verbose --help | egrep ^datadir
datadir /var/lib/mysql/
ls -ld /var/lib/mysql
lrwxrwxrwx 1 mysql mysql 27 Mar 7 14:13 /var/lib/mysql -> /mnt/rwstore/rwstore/mysql/

Robie Basak (racb) on 2016-08-02
tags: added: mysql-alternate-datadir
Robie Basak (racb) on 2016-08-05
Changed in mysql-5.7 (Ubuntu):
milestone: ubuntu-16.06 → ubuntu-16.09
ironstorm (ironstorm-gmail) wrote :

If there are no plans to fix this, can we just stop releasing new updates of this package?

I personally don't need the unrelated mysql fixes and they just require us to re-patch mysql postinst again and again so the other dependent packages with actual fixes can satisfy their requirement for the latest mysql package revision.

Agree its frustrating that links don't work, especially as they hardcode directories in /usr/share/mysql/mysql-systemd-start (see bug/enhancement request #1611018)

Lars Tangvald (lars-tangvald) wrote :

Sorry, yes, this should have been fixed a long time ago

Changed in mysql-5.7 (Ubuntu):
assignee: nobody → Lars Tangvald (lars-tangvald)
status: Confirmed → In Progress
asavah (irherder) wrote :

mysql-server-5.7_5.7.15-0ubuntu0.16.04.1 arrived with critical security fixes,
but this ONE LETTER bug is still not fixed.

How much do you want for fixing the bug that has a known fix FFS?
I'm willing to pay ...

Lars Tangvald (lars-tangvald) wrote :

We should have this fixed for the September milestone

Lars Tangvald (lars-tangvald) wrote :

Also, from the mysql-server-5.7_5.7.15-0ubuntu0.16.04.1 changelog:
  * SECURITY UPDATE: Update to 5.7.15 to fix security issues
    - CVE-2016-6662

:)

Changed in mysql-5.7 (Ubuntu):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mysql-5.7 - 5.7.15-0ubuntu2

---------------
mysql-5.7 (5.7.15-0ubuntu2) yakkety; urgency=medium

  * Add working dir to perl lib path for dep8 upstream.
    New versions of perl will no longer automatically include working dir in
    the path. This was causing the mtr suite to fail to start.
  * Change mysql-testsuite dependency from python to libjson-perl.
    Tests written in python were rewritten in perl, so testsuite no longer
    depends on python, but tests fail if libjson-perl is missing.
  * Build with DWITH_LZ4=system
    If the flag is not set the package will be built with a bundled liblz4
    instead of the one provided by the liblz4-dev build-dep.
  * Add support for custom datadir to systemd service (LP: #1574782)
    The service was reporting errors if no database was found in /var/lib/mysql
    It will now check the path specified in the config
  * Fix support for soft-link datadir (LP: #1474212)
    Upgrading more than once when datadir is soft-link was causing errors
  * Escape special characters in password (LP: #1598992)
    The password string was not escaped, causing SQL errors if certain special
    characters such as apostrophes were included.
  * Removed always-false check on purge in postrm (LP: #1602945)
    Users were never seeing the query to remove data on a package purge

 -- Lars Tangvald <email address hidden> Fri, 16 Sep 2016 22:23:41 +0200

Changed in mysql-5.7 (Ubuntu):
status: Fix Committed → Fix Released
tags: added: xenial-candidate
Robie Basak (racb) wrote :

Otto, I'm not sure if this affects mariadb-10.0 on Xenial.

Changed in mysql-5.7 (Ubuntu Xenial):
status: New → Triaged
importance: Undecided → Medium
xylo (stefan-endrullis) wrote :

I still have this bug with mysql-server-5.7_5.7.15-0ubuntu0.16.04.1_amd64.deb downloaded today.

In my case /var/lib/mysql is a symlink that links to another partition.

Lars Tangvald (lars-tangvald) wrote :

xylo: Yes, we haven't ported it to Xenial yet. Note that there's a separate issue with having datadir set to the root of a partition (LP: #1580794), but that only affects fresh installs

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mariadb-10.0 (Ubuntu Xenial):
status: New → Confirmed
description: updated
Shawn (shawnpatti) wrote :

A bind mount rather than a link will resolve the issue.

Changed in mysql-5.7 (Ubuntu Xenial):
assignee: nobody → Lars Tangvald (lars-tangvald)
status: Triaged → Fix Committed
Changed in mysql-5.7 (Ubuntu Xenial):
status: Fix Committed → In Progress
Robie Basak (racb) on 2017-01-06
Changed in mysql-5.7 (Ubuntu Xenial):
milestone: none → xenial-updates
Johan Zetterberg (johanz) wrote :

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

$ sudo apt-get update && sudo apt-get upgrade
Hit:1 http://se.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://se.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:3 http://se.archive.ubuntu.com/ubuntu xenial-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run 'apt-get -f install' to correct these.
The following packages have unmet dependencies:
 mysql-server-5.7 : Depends: mysql-server-core-5.7 (= 5.7.16-0ubuntu0.16.04.1) but 5.7.17-0ubuntu0.16.04.1 is installed
E: Unmet dependencies. Try using -f.

$ sudo apt-get upgrade -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
  linux-image-extra-4.4.0-45-generic
Use 'sudo apt autoremove' to remove it.
The following packages will be upgraded:
  mysql-server-5.7
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
78 not fully installed or removed.
Need to get 0 B/2 856 kB of archives.
After this operation, 92,2 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Preconfiguring packages ...
(Reading database ... 166731 files and directories currently installed.)
Preparing to unpack .../mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb ...
cp: cannot overwrite directory '/var/lib/mysql-upgrade/DATADIR.link/mysql' with non-directory
dpkg: error processing archive /var/cache/apt/archives/mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Hello itsadok, or anyone else affected,

Accepted mysql-5.7 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mysql-5.7/5.7.17-0ubuntu0.16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in mysql-5.7 (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Lars Tangvald (lars-tangvald) wrote :

Verified fixed in 5.7.17-0ubuntu0.16.04.2

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mysql-5.7 - 5.7.17-0ubuntu0.16.04.2

---------------
mysql-5.7 (5.7.17-0ubuntu0.16.04.2) xenial; urgency=medium

  * Add libjson-perl dependency for test suite (LP: #1631338)
    The last two python tests in the mtr suite were rewritten
    in perl, and require this package. Added for mysql-testsuite
    and as a build-dep.
  * Build with DWITH_LZ4=system (LP: #1631339)
    Package was previously built with bundled liblz4, though it
    was specified as a build-dep.
  * Add support for custom datadir to systemd service (LP: #1574782)
    The service was reporting an error if no database could be found in
    /var/lib/mysql. It now checks the location specified in the config.
    Note that user must still handle apparmor access for custom datadir.
  * Fix copy of soft-link datadir to /var/lib/mysql-upgrade (LP: #1474212)
    Upgrade would sometimes fail if mysql-upgrade already contained a
    link copy from a previous upgrade.
  * Escape special characters in password (LP: #1598992)
    Special characters in the root password would cause syntax errors
    and postinst failures.
  * Failing tests on platforms supported by upstream fail build (LP: #1646488)
    A passing test suite is now enforced on i386 and amd64 platforms at
    build-time.
  * d/copyright: Updated with information about new source files
  * Unstable test main.xa_prepared_binlog_off disabled pending upstream fix.
    Upstream bug report: http://bugs.mysql.com/bug.php?id=83340
  * d/lintian-overrides: Updated line numbers

 -- Lars Tangvald <email address hidden> Fri, 03 Feb 2017 08:53:04 +0100

Changed in mysql-5.7 (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for mysql-5.7 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Faustin (fauust) on 2018-02-15
Changed in mariadb-10.0 (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.