mysql reconnect does not catch 1047 errors

[Sergey Vasilenko]

Any service using oslo db reconnect feature does not catch 1047 mysqldb error which happens after galera cluster is in non-primary state. this can lead cluster being non-functional even after galera functioning is restored.

[Vladimir Kuklin (vkuklin)]

it shows that our db reconnect patch does not catch 1047 mysqlDB error. we need to fix it

[Vladimir Kuklin (vkuklin)]

this should be used done fuel-hardening team.

[Roman Vyalov (r0mikiam)]

need get patches of old git branch (openstack-ci/fuel/havana) and push to fuel-5.0 branch

[Max Rasskazov (mrasskazov)]

ceilometer: https://gerrit.mirantis.com/14974
nova: https://gerrit.mirantis.com/14979
neutron: https://gerrit.mirantis.com/14980
keystone: https://gerrit.mirantis.com/14980
heat: https://gerrit.mirantis.com/14982
glance: https://gerrit.mirantis.com/14984
cinder: https://gerrit.mirantis.com/14986
sahara: https://gerrit.mirantis.com/14990
murano-api: https://gerrit.mirantis.com/14991

[Max Rasskazov (mrasskazov)]

keystone: https://gerrit.mirantis.com/14981

[Meg McRoberts (dreidellhasa)]

Added to fixed issues in 5.0 Release Notes.

[Alexander Ignatov (aignatov)]

Reopened this bug because patch https://review.fuel-infra.org/#/c/93/ could not be applied since all components switched to oslo.db . Thus we need to check if oslo.db contains this bug or not.

[OSCI Robot (oscirobot)]

Package oslo.db has been built from changeset: https://review.fuel-infra.org/220
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-stable-220/ubuntu

[OSCI Robot (oscirobot)]

Package oslo.db has been built from changeset: https://review.fuel-infra.org/220
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-stable-220/centos

[OSCI Robot (oscirobot)]

Package oslo.db has been built from changeset: https://review.fuel-infra.org/220
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-stable-220/ubuntu

[OSCI Robot (oscirobot)]

Package oslo.db has been built from changeset: https://review.fuel-infra.org/220
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-stable-220/centos

[OSCI Robot (oscirobot)]

Package oslo.db has been built from changeset: https://review.fuel-infra.org/220
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-stable/centos

[OSCI Robot (oscirobot)]

Package oslo.db has been built from changeset: https://review.fuel-infra.org/220
DEB Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-stable/ubuntu

[Viktor Serhieiev (vsergeyev)]

Can someone provide steps to reproduce this bug?

[Roman Podoliaka (rpodolyaka)]

Fixed in 6.0: https://review.fuel-infra.org/gitweb?p=openstack/oslo.db.git;a=commit;h=c4a4cc3ffac08aa2ea2c900423e994cbd3453c11

[Alexander Gubanov (ogubanov)]

I checked code on mos6.0 (build 56) - fixed

[root@nailgun ~]# fuel --f
api: '1.0'
astute_sha: 16b252d93be6aaa73030b8100cf8c5ca6a970a91
auth_required: true
build_id: 2014-12-18_01-32-01
build_number: '56'
feature_groups:
- mirantis
fuellib_sha: 73332192a257ea02c40a39885c502ad1ebdf3eda
fuelmain_sha: 45caacadb878abfbd9d60e134d72229698b469c9
nailgun_sha: 5f91157daa6798ff522ca9f6d34e7e135f150a90
ostf_sha: a9afb68710d809570460c29d6c3293219d3624d4
production: docker
release: '6.0'
release_versions:
  2014.2-6.0:
    VERSION:
      api: '1.0'
      astute_sha: 16b252d93be6aaa73030b8100cf8c5ca6a970a91
      build_id: 2014-12-18_01-32-01
      build_number: '56'
      feature_groups:
      - mirantis
      fuellib_sha: 73332192a257ea02c40a39885c502ad1ebdf3eda
      fuelmain_sha: 45caacadb878abfbd9d60e134d72229698b469c9
      nailgun_sha: 5f91157daa6798ff522ca9f6d34e7e135f150a90
      ostf_sha: a9afb68710d809570460c29d6c3293219d3624d4
      production: docker
      release: '6.0'

root@node-7:~# grep '1047' /usr/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py
         r".*\((?:2002|2003|2006|2013|1047)")

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Dina Belova <email address hidden>
Review: https://review.fuel-infra.org/8272

[Davanum Srinivas (DIMS) (dims-v)]

https://review.openstack.org/#/c/201706/

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/8272
Submitter: Victor Sergeyev <email address hidden>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: fd761f6916f7111ded68768e27dcdf29bc551bc7
Author: Dina Belova <email address hidden>
Date: Thu Jul 23 08:32:54 2015

Added catching of errors 1047 (Galera) for MySQL oslo db reconnect

Fixed H238 flake8 errors, which were raised because of the new hacking

Change-Id: I03ae2ca87dfe02a7838576b17dc707b80d199c2c
Closes-Bug: #1312212