Disappearing /var/run/mysqld causes mysqld to fail after reboot

[Luca D'Isanto]

This bug is no longer a duplicate of bug 1421303
(I open thi bug because Robie Basak say to open new bug)

with mysql-server version: 5.6.23-1~exp1~ubuntu4 mysql.service fail to start after reboot:

● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
   Active: activating (start-post) since mar 2015-03-24 08:41:00 CET; 25s ago
  Process: 951 ExecStart=/usr/bin/mysqld_safe (code=exited, status=0/SUCCESS)
  Process: 943 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
 Main PID: 951 (code=exited, status=0/SUCCESS); : 952 (mysql-systemd-s)
   CGroup: /system.slice/mysql.service
           └─control
             ├─ 952 /bin/bash /usr/share/mysql/mysql-systemd-start post
             └─2601 sleep 1

mar 24 08:41:00 luca-k53sd systemd[1]: Starting MySQL Community Server...
mar 24 08:41:00 luca-k53sd mysqld_safe[951]: 150324 08:41:00 mysqld_safe Can't log to error log and syslog at the same time. Remove all --log-err...e effect.
mar 24 08:41:00 luca-k53sd mysqld_safe[951]: 150324 08:41:00 mysqld_safe Logging to '/var/log/mysql/error.log'.
mar 24 08:41:00 luca-k53sd mysqld_safe[951]: mkdir: impossibile creare la directory "/var/run/mysqld": Permesso negato
mar 24 08:41:00 luca-k53sd mysqld_safe[951]: chown: impossibile accedere a "/var/run/mysqld": File o directory non esistente
mar 24 08:41:00 luca-k53sd mysqld_safe[951]: chmod: impossibile accedere a "/var/run/mysqld": File o directory non esistente
mar 24 08:41:00 luca-k53sd mysqld_safe[951]: 150324 08:41:00 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Hint: Some lines were ellipsized, use -l to show in full.

ProblemType: Bug
DistroRelease: Ubuntu 15.04
Package: mysql-server 5.6.23-1~exp1~ubuntu4
ProcVersionSignature: Ubuntu 3.19.0-10.10-generic 3.19.2
Uname: Linux 3.19.0-10-generic x86_64
ApportVersion: 2.16.2-0ubuntu4
Architecture: amd64
CurrentDesktop: KDE
Date: Tue Mar 24 13:05:23 2015
InstallationDate: Installed on 2015-03-18 (6 days ago)
InstallationMedia: Kubuntu 15.04 "Vivid Vervet" - Alpha amd64 (20150318)
PackageArchitecture: all
SourcePackage: mysql-5.6
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Bruno Patri (bruno666-deactivatedaccount)]

Create a newfile /etc/tmpfiles.d/mysql.conf:

# systemd tmpfile settings for mysql
# See tmpfiles.d(5) for details

d /var/run/mysqld 0755 mysql mysql -

After reboot, mysql should start normally.

[Robie Basak (racb)]

@Luca

Does Bruno's fix work for you? I'd like to know if so, because that suggests that we have a problem with /var/run/mysqld not recreated after boot which is certainly a bug. I need to figure out steps to reproduce in order to fix it.

[Luca D'Isanto (lukadisanto)]

Hi,
With Bruno's fix works

Thank you!!

[Bruno Patri (bruno666-deactivatedaccount)]

@Luca: prego;)

I guess that /run/mysqld is not created after boot because in /lib/systemd/mysql.service, mysqld_safe script (which seems to be responsible for its creation) is invoked with user/group mysql/mysql and /run is only writable by root.

[Bruce Pieterse (octoquad)]

Comment #3 works for me as well.

[Rocko (rockorequin)]

Comment #3 fixed it for me, too.

[dave nelson (davidnelson)]

Comment #3 fixed it for me, too.

[atomiko (lex-atomiko)]

Comment #3 works perfect, thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package mysql-5.6 - 5.6.23-1~exp1~ubuntu5

---------------
mysql-5.6 (5.6.23-1~exp1~ubuntu5) vivid; urgency=medium

  * Ubuntu upload from Debian VCS commit 804459b.

mysql-5.6 (5.6.23-1~exp1) UNRELEASED; urgency=medium

  [ Robie Basak ]
  * New upstream release.

  [ Norvald H. Ryeng ]
  * Remove unused build options.
  * Add libedit dependency to mysql-client-core-5.6.
  * Remove chmods not needed after fix for upstream bug #68517.
  * Lintian fixes.

  [ Bjoern Boschman ]
  * Removed hardening-wrapper from build-deps.
  * Added python dep for mysql-testsuite-5.6.
  * Added dpkg-dev dep.
  * Cleanup a bit more.

  [ Robie Basak ]
  * Add dep8 smoke test.
  * Re-enable libmysqld-pic, libmysqld-dev, libmysqlclient18 and
    libmysqlclient-dev (renamed accordingly) to replace binary packages
    provided by src:mysql-5.5.
  * Add mysql-server, mysql-client and mysql-testsuite to be switched
    over from src:mysql-5.5.
  * Import mysql-common from src:mysql-5.5 for 5.5 -> 5.6 switchover.
  * Cherry-pick relevant parts of Ubuntu delta from mysql-5.5:
    - d/mysql-server-5.5.postrm: restoring postrm debhelper token to get
      proper behaviors from dh_installinit and dh_apparmor.
    - Add Apport hook.

  [ James Page ]
  * d/tests/upstream: Add main.ctype_uca to skip-test list as it requires
    writable /usr.
  * d/p/fix-mysqlhotcopy-test-failure.patch: Add return code 255 to the
    list of allowed return codes.
  * d/tests/control: Re-enable upstream DEP-8 tests.

  [ Otto Kekäläinen ]
  * Added parallel build support via DEB_BUILD_OPTIONS

  [ Otto Kekäläinen ]
  * Remove makeflags from cmake command and output MAKEFLAGS for debugging
    purposes just before make is run

  [ Akhil Mohan ]
  * Provisioned dep8 test case upstream separated to run as non-root

  [ Norvald H. Ryeng ]
  * Simplify and stabilize tests
  * Remove USE_STATIC_MYSQLD
  * Add test plan to README.Maintainer
  * Add --force rationale to test plan

  [ Akhil Mohan ]
  * Make builds parallel Use MAKEFLAGS for parallel builds

  [ Otto Kekäläinen ]
  * Move mysql_plugin to server package as it controls the server plugins
  * Wrap-and-sort
  * Remove legacy 5.0->5.1 transition code that refers a internal-use-only
    -directory that does not even exist anymore.
  * Removed all unnecessary .dirs as most of the directories are generated
    automatically
  * Don't emit extra output at init invocation Many users have complained that
    they're database is corrupt because they tried to start the database and it
    refuses to do so, and at the same time emitting a message with the word
    "corrupt". this is just plain wrong and confusing, so I'm removing it.

  [ Akhil Mohan ]
  * added systemd service profile and script

  [ Norvald H. Ryeng ]
  * Remove special build options for taocrypt

  [ Robie Basak ]
  * Drop unused variable definition
  * Create /etc/mysql manually in debian/rules
  * Replace conf.d/.keepme with an empty directory

  [ Akhil Mohan ]
  * Standards version moved up to 3.9.6 in d/control
  * Added Systemd service profile and script

  [ Robie Basak ]
  * wrap-and-sort for systemd change
  * Drop unnecessary binary target
  ...

[al (e-webmaszer-t)]

Hi,

I am not an expert,so I do'nt know how to apply the Bruno's fix

When I start mysql I get:

Sep 11 11:08:52 serv mysql[25802]: Checking for tables which need an up...e
Sep 11 11:08:52 serv mysql[25802]: not closed cleanly..
Hint: Some lines were ellipsized, use -l to show in full.

I understand I have to

Create a newfile /etc/tmpfiles.d/mysql.conf:

What does it mean " d /var/run/mysqld 0755 mysql mysql - " ? Delete?

Thank You to all mysql experts that can solve this question.

Yours Sincerely,
Alex

[Bruce Pieterse (octoquad)]

Hi al,

As per tempfile(1) (man tempfile) that line creates a new directory at /var/run/mysqld with 0755 permissions and sets the owner and group to mysql.

[Robie Basak (racb)]

@kiaikim

Why did you add a task for mysql-5.7? This bug was fixed in 5.6.25-2 which predates mysql-5.7.

[kiaikim (kiaikim)]

@racb
I'm sorry. I think I got webpages confused. I had the same problem with 5.7.17 on Ubuntu 16.04. I looked all over and tried whatever I could find. Must have thought I tried #3, but I did chown of /var/run/mysqld/ with mysql service running, which also worked. If you'd like I can start over, see if the problem persists and try #3. But probably better to see if others have this problem.

[kiaikim (kiaikim)]

Actually, I created /var/run/mysqld/, and then did chown.

[Robie Basak (racb)]

No problem. Feel free to file a separate bug against mysql-5.7 and link to it from here. If you can, please find steps that a developer can use to reproduce the problem. If you can't, then feel free to file a bug but please set the status to Incomplete. Others affected can then find and communicate in the bug with you to try and figure out steps to reproduce, or a technical explanation of what is happening.

Note though that the community doesn't generally follow bugs, so to find others to get help you are more likely to get help in other places. See http://www.ubuntu.com/support/community for community support options.

[Orin Fink (orin-fink)]

This issue seems to be relevant also in the most recent update from apt for Ubuntu 14.04. (Note i am running percona version) I just ran the update this morning and am now seeing this same issue. I've tried to implement the tmfiles.d solution but that does not seem to take effect. I _can_ manually create the /var/run/mysqld/ folder and chown same as kiakim, but this manual step is a bit wonky.

When will the 'fix' be available? Or, is this now an issue for the percona team to address?

[Orin Fink (orin-fink)]

This may or may not be relevant... but i did notice in the release notes for the 5.6.35 version of mysql (the version now being used by our ubuntu 14.04 update)... it looks like work has been done on the mysqld_safe script and use of chown is removed. Is it possible the necessary security lockdown re-introduced the bug.

https://dev.mysql.com/doc/relnotes/mysql/5.6/en/news-5-6-35.html

[Szalai Kálmán (ulyszk)]

Same issue here, workaround by tmpfiles.d.

[Robie Basak (racb)]

> When will the 'fix' be available?

This bug has already been fixed. If the issue has somehow recurred, or occurred in a different package, then please file a new bug with a full report (packages and versions used, steps to reproduce, etc) and link to this one. If you do not do this, it will not appear in our reporting and thus will never get fixed.

[kiaikim (kiaikim)]

I'm not the only one having this problem with 5.7. See the first comment under http://stackoverflow.com/a/42640731/2668783

[Robie Basak (racb)]

> I'm not the only one having this problem with 5.7.

That's fine, but as I said, please file a separate bug with a full report.