Liberty to Liberty Upgrade: MariaDB upgrade stuck on post dpkg configure task
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack-Ansible |
Won't Fix
|
Undecided
|
Bjoern |
Bug Description
While upgrading from OSA 12.0.15 to 12.2.8 I noticed that mariadb got stuck during upgrade at the Galera Server start task :
infra01:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 33236 3832 ? Ss 13:02 0:00 /sbin/init
root 267 0.0 0.0 19480 180 ? S 13:02 0:00 upstart-udev-bridge --daemon
message+ 270 0.0 0.0 39116 320 ? Ss 13:02 0:00 dbus-daemon --system --fork
root 299 0.0 0.0 49272 3220 ? Ss 13:02 0:00 /lib/systemd/
syslog 319 0.0 0.0 405568 2756 ? Ssl 13:02 0:00 rsyslogd
root 346 0.0 0.0 15412 1708 ? S 13:02 0:00 upstart-file-bridge --daemon
root 349 0.0 0.0 15264 212 ? S 13:02 0:00 upstart-
root 385 0.0 0.0 10224 4208 ? Ss 13:02 0:00 dhclient -1 -v -pf /run/dhclient.
root 491 0.0 0.0 12788 1936 lxc/tty4 Ss+ 13:02 0:00 /sbin/getty -8 38400 tty4
root 494 0.0 0.0 12788 1944 lxc/tty2 Ss+ 13:02 0:00 /sbin/getty -8 38400 tty2
root 495 0.0 0.0 12788 1960 lxc/tty3 Ss+ 13:02 0:00 /sbin/getty -8 38400 tty3
root 509 0.0 0.0 23656 2324 ? Ss 13:02 0:00 cron
root 515 0.0 0.0 61380 5424 ? Ss 13:02 0:00 /usr/sbin/sshd -D
root 557 0.0 0.0 18236 3140 ? S 13:02 0:00 /bin/bash /usr/bin/
mysql 980 48.9 0.8 9022584 3463372 ? Sl 13:03 14:30 /usr/sbin/mysqld --basedir=/usr --datadir=
root 1133 0.0 0.0 12788 1908 lxc/console Ss+ 13:03 0:00 /sbin/getty -8 38400 console
root 1137 0.0 0.0 12788 1880 lxc/tty1 Ss+ 13:03 0:00 /sbin/getty -8 38400 tty1
root 2461 0.0 0.0 93340 6400 ? Ss 13:26 0:00 sshd: root@notty
root 3796 0.0 0.0 4448 688 ? Ss 13:27 0:00 /bin/sh -c LANG=en_US.UTF-8 LC_CTYPE=
root 3797 0.0 0.0 99696 53536 ? S 13:27 0:00 /usr/bin/python
root 3801 3.2 0.0 73620 38596 ? S 13:27 0:10 /usr/bin/apt-get -y -o Dpkg::Options:
root 3808 0.0 0.0 18304 3432 ? S 13:27 0:00 /bin/bash
root 4324 0.0 0.0 13476 3472 ? S 13:27 0:00 /usr/bin/dpkg --force-confdef --force-confold --status-fd 43 --configure galera-3:amd64 mariadb-
root 4325 0.0 0.0 58320 13536 ? S 13:27 0:00 /usr/bin/perl -w /usr/share/
root 4331 0.0 0.0 9680 2792 ? S 13:27 0:00 /bin/bash -e /var/lib/
mysql 4474 0.4 0.1 5024208 580608 ? Sl 13:31 0:00 /usr/sbin/mysqld --bootstrap --user=mysql --disable-log-bin --skip-grant-tables --default-
root 4495 0.0 0.0 15568 2108 ? R+ 13:32 0:00 ps uax
To overcome this issue I had to manually stop the database, the playbooks continued at this point.
Additionally I noticed that mysql_install_db is running as part of the dpkg post install on an existing DB, as here shown for infra02
root 3508 0.0 0.0 4448 768 ? Ss 13:34 0:00 /bin/sh -c LANG=en_US.UTF-8 LC_CTYPE=
root 3509 0.6 0.0 99700 53392 ? S 13:34 0:00 /usr/bin/python
root 3513 3.4 0.0 73620 38428 ? S 13:34 0:01 /usr/bin/apt-get -y -o Dpkg::Options:
root 4026 0.0 0.0 13476 3452 ? S 13:34 0:00 /usr/bin/dpkg --force-confdef --force-confold --status-fd 43 --configure galera-3:amd64 mariadb-
root 4027 0.3 0.0 58320 13428 ? S 13:34 0:00 /usr/bin/perl -w /usr/share/
root 4033 0.0 0.0 9672 2752 ? S 13:34 0:00 /bin/bash -e /var/lib/
root 4072 0.0 0.0 9640 2704 ? S 13:34 0:00 /bin/bash /usr/bin/
root 4073 0.0 0.0 4344 1420 ? S 13:34 0:00 logger -p daemon.err -t mysqld_safe -i
root 4157 0.0 0.0 9640 1800 ? S 13:35 0:00 /bin/bash /usr/bin/
mysql 4159 14.5 0.1 5237520 600636 ? Sl 13:35 0:00 /usr/sbin/mysqld --lc-messages=en_US --bootstrap --basedir=/usr --datadir=
root 4186 0.0 0.0 15568 2120 pts/0 R+ 13:35 0:00 ps aux
which concerns me and that maybe the reason why the upgrade got stuck in the first place.
Since Liberty is now EOL, I'm not sure how we handle this issue which may be reappearing on future versions. That's why I opened this issue
As you may see, the galera version updated from 10.0.27 to 10.0.28