Arbitrary OSTF failures due to "1047 WSREP has not yet prepared node for application use" from Galera

Bug #1654321 reported by Roman Podoliaka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Confirmed
High
Fuel Sustaining

Bug Description

Recently we had a few similar BVT failures:

https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/1041
https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/1042
https://product-ci.infra.mirantis.net/view/10.0/job/10.0.main.ubuntu.bvt_2/1104/

In application logs (neutron , nova, keystone, etc) there is the following error:

2017-01-04 16:46:22.335 16953 ERROR nova.api.openstack.extensions InternalError: (pymysql.err.InternalError) (1047, u'WSREP has not yet prepared node for application use')

It's not clear why it happens as the Galera cluster was assembled properly and worked normally for some time (most OSTF tests passed).

MySQL OCF script logs from one of the nodes (is it ok to have master re-elections that often?):

http://xsnippet.org/362211/raw/

Tags: area-linux
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :
Changed in mos:
assignee: nobody → Sergii Golovatiuk (sgolovatiuk)
Changed in mos:
importance: Undecided → High
Changed in mos:
status: New → Confirmed
milestone: none → 10.0
tags: added: area-linux
Changed in mos:
assignee: Sergii Golovatiuk (sgolovatiuk) → Fuel Sustaining (fuel-sustaining-team)
Revision history for this message
Vitalii Gridnev (vgridnev) wrote :

Another failure: https://ci.fuel-infra.org/job/10.0-community.main.ubuntu.bvt_2/1132/

 grep -nr " u'WSREP has not yet prepared node for application use'" . | wc -l returns 819 lines.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.