Comment 0 for bug 1827690

Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

After deploying barbican (stein, 19.04 charms) I got into a situation where both workers (two-node HA) are down due to a failed alembic migration:

https://private-fileshare.canonical.com/~dima/charm-dumps/barbican-0-var-log.tar.gz

May 03 20:51:11 juju-35b20e-3-lxd-0 systemd[1]: Started OpenStack Barbican Key Management Workers.
May 03 20:51:39 juju-35b20e-3-lxd-0 systemd[1]: barbican-worker.service: Main process exited, code=exited, status=1/FAILURE
May 03 20:51:39 juju-35b20e-3-lxd-0 systemd[1]: barbican-worker.service: Failed with result 'exit-code'.

juju status barbican
Model Controller Cloud/Region Version SLA Timestamp
openstack samaas samaas 2.6-rc1 unsupported 23:27:23Z

App Version Status Scale Charm Store Rev OS Notes
barbican 8.0.0 blocked 2 barbican jujucharms 18 ubuntu
barbican-vault 1.2.2 active 2 barbican-vault jujucharms 2 ubuntu
hacluster-barbican active 2 hacluster jujucharms 54 ubuntu

Unit Workload Agent Machine Public address Ports Message
barbican/0 blocked idle 3/lxd/0 10.232.46.135 9311/tcp,9312/tcp Services not running that should be: barbican-worker
  barbican-vault/1 active idle 10.232.46.135 Unit is ready
  hacluster-barbican/1 active idle 10.232.46.135 Unit is ready and clustered
barbican/1* blocked idle 2/lxd/0 10.232.46.130 9311/tcp,9312/tcp Services not running that should be: barbican-worker
  barbican-vault/0* active idle 10.232.46.130 Unit is ready
  hacluster-barbican/0* active idle 10.232.46.130 Unit is ready and clustered

https://paste.ubuntu.com/p/BS3fHw287r/
2019-05-03 21:13:46.582 115638 ERROR barbican alembic.util.exc.CommandError: Requested revision 1a0c2cdafb38 overlaps with other requested revisions 39cf2e645cba

mysql> use barbican;
mysql> select * from alembic_version;
+--------------+
| version_num |
+--------------+
| 1a0c2cdafb38 |
| 39cf2e645cba |
+--------------+
2 rows in set (0.00 sec)