Wait_for_backend[mysql] runs before Xinetd::Service[galeracheck] on primary controller

Bug #1586390 reported by Sergii Golovatiuk
22
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Sergii Golovatiuk
Mitaka
Fix Released
High
Maciej Relewicz

Bug Description

https://review.openstack.org/#/c/315046/ introduced a regression where backend runs before xinetd configuration. However, it doesn't fail deployment as task will be rerun so backend will be configured on next run. Though it's not handy for UX experience

How to reproduce:

Check logs for primary controller, they contain

2016-05-27T11:07:11.578578+00:00 notice: (/Stage[main]/Cluster::Galera_status/Augeas[galeracheck]/returns) executed successfully
2016-05-27T11:07:11.581097+00:00 notice: (/Stage[main]/Cluster::Galera_status/Xinetd::Service[galeracheck]/File[/etc/xinetd.d/galeracheck]/ensure) created
2016-05-27T11:17:11.941612+00:00 err: Timeout waiting for HAProxy backend: 'mysqld' status to become: 'up' after 600 seconds!
2016-05-27T11:17:11.941612+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'
2016-05-27T11:17:11.941612+00:00 err: /etc/puppet/modules/haproxy/lib/puppet/provider/haproxy_backend_status/haproxy.rb:72:in `ensure='
...
2016-05-27T11:17:11.943897+00:00 err: (/Stage[main]/Osnailyfacter::Database::Database/Osnailyfacter::Wait_for_backend[mysql]/Haproxy_backend_status[mysql]/ensure) change from down to up failed: Timeout waiting for HAProxy backend: 'mysqld' status to become: 'up' after 600 seconds!
2016-05-27T11:17:11.983333+00:00 notice: (/Stage[main]/Xinetd/Service[xinetd]) Triggered 'refresh' from 2 events
2016-05-27T11:17:12.107719+00:00 notice: Finished catalog run in 655.19 seconds

Expected: clean log for database task without any errors.

Changed in fuel:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Maciej Relewicz (rlu)
milestone: none → 10.0
Changed in fuel:
status: Triaged → In Progress
tags: added: area-library
Revision history for this message
Oleksiy Molchanov (omolchanov) wrote :
Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Marking as a swarm-blocker because one of duplicates has this tag.

tags: added: swarm-blocker
Changed in fuel:
assignee: Maciej Relewicz (rlu) → Sergii Golovatiuk (sgolovatiuk)
Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/323323

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/322157
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=cba4ab766a41859295413640595923effaa3490c
Submitter: Jenkins
Branch: master

commit cba4ab766a41859295413640595923effaa3490c
Author: Maciej Relewicz <email address hidden>
Date: Fri May 27 15:00:54 2016 +0200

    Add contain for proper relationship between classes

    Change-Id: I1e1cbfae716ec4828bd64e918e5e6baae61966ad
    Closes-Bug: #1586390
    Signed-off-by: Sergii Golovatiuk <email address hidden>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/mitaka)

Reviewed: https://review.openstack.org/323323
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=b24fafbadbd907be3a10946278f5785bb354d17a
Submitter: Jenkins
Branch: stable/mitaka

commit b24fafbadbd907be3a10946278f5785bb354d17a
Author: Maciej Relewicz <email address hidden>
Date: Fri May 27 15:00:54 2016 +0200

    Add contain for proper relationship between classes

    Change-Id: I1e1cbfae716ec4828bd64e918e5e6baae61966ad
    Closes-Bug: #1586390
    Signed-off-by: Sergii Golovatiuk <email address hidden>

Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

Please, check attached logs, it's still failing
steps are described here - https://bugs.launchpad.net/fuel/+bug/1586402

Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :
Changed in fuel:
status: Fix Committed → New
Revision history for this message
Sergii Golovatiuk (sgolovatiuk) wrote :

Please analyze logs before reopening a case. The last attachment shows completely different issue

Changed in fuel:
status: New → Fix Committed
tags: added: on-verification
Revision history for this message
Dmitry Belyaninov (dbelyaninov) wrote :

Verified. There are no described log messages.

[root@nailgun ~]# shotgun2 short-report
cat /etc/fuel_build_id:
 455
cat /etc/fuel_build_number:
 455
cat /etc/fuel_release:
 9.0
cat /etc/fuel_openstack_version:
 mitaka-9.0
rpm -qa | egrep 'fuel|astute|network-checker|nailgun|packetary|shotgun':
 fuel-release-9.0.0-1.mos6347.noarch
 fuel-misc-9.0.0-1.mos8443.noarch
 python-packetary-9.0.0-1.mos140.noarch
 fuel-bootstrap-cli-9.0.0-1.mos285.noarch
 fuel-migrate-9.0.0-1.mos8443.noarch
 rubygem-astute-9.0.0-1.mos749.noarch
 fuel-provisioning-scripts-9.0.0-1.mos8735.noarch
 fuel-mirror-9.0.0-1.mos140.noarch
 shotgun-9.0.0-1.mos90.noarch
 fuel-openstack-metadata-9.0.0-1.mos8735.noarch
 fuel-notify-9.0.0-1.mos8443.noarch
 nailgun-mcagents-9.0.0-1.mos749.noarch
 python-fuelclient-9.0.0-1.mos324.noarch
 fuel-9.0.0-1.mos6347.noarch
 fuel-utils-9.0.0-1.mos8443.noarch
 fuel-setup-9.0.0-1.mos6347.noarch
 fuel-library9.0-9.0.0-1.mos8443.noarch
 network-checker-9.0.0-1.mos74.x86_64
 fuel-agent-9.0.0-1.mos285.noarch
 fuel-ui-9.0.0-1.mos2716.noarch
 fuel-ostf-9.0.0-1.mos935.noarch
 fuelmenu-9.0.0-1.mos272.noarch
 fuel-nailgun-9.0.0-1.mos8735.noarch
[root@nailgun ~]#

tags: removed: on-verification
Revision history for this message
Ekaterina Shutova (eshutova) wrote :

Verified on 10.0 #1578.
Checked logs on primary-controller and there are no messages from description.

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.