Add a base constraint so gnocchi services should start after galera

Bug #1604860 reported by Pradeep Kilambi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Undecided
Pradeep Kilambi

Bug Description

When we restart the cluster sometime statsd comes up before galera causing connection issues and fails. Add a constraint to order so statsd starts after openstack-core-clone.

Relevant traceback from yurii:

2016-07-20 11:57:54.399 6045 CRITICAL gnocchi [-] StorageError: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-07-20 11:57:54.399 6045 ERROR gnocchi Traceback (most recent call last):
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/bin/gnocchi-statsd", line 10, in <module>
2016-07-20 11:57:54.399 6045 ERROR gnocchi sys.exit(statsd())
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/cli.py", line 75, in statsd
2016-07-20 11:57:54.399 6045 ERROR gnocchi statsd_service.start()
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/statsd.py", line 174, in start
2016-07-20 11:57:54.399 6045 ERROR gnocchi stats = Stats(conf)
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/statsd.py", line 38, in __init__
2016-07-20 11:57:54.399 6045 ERROR gnocchi self.storage = storage.get_driver(self.conf)
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/storage/__init__.py", line 155, in get_driver
2016-07-20 11:57:54.399 6045 ERROR gnocchi return get_driver_class(conf)(conf.storage)
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/storage/ceph.py", line 61, in __init__
2016-07-20 11:57:54.399 6045 ERROR gnocchi super(CephStorage, self).__init__(conf)
2016-07-20 11:57:54.399 6045 ERROR gnocchi File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 61, in __init__
2016-07-20 11:57:54.399 6045 ERROR gnocchi raise storage.StorageError("Unable to start coordinator: %s" % e)
2016-07-20 11:57:54.399 6045 ERROR gnocchi StorageError: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-07-20 11:57:54.399 6045 ERROR gnocchi

Changed in tripleo:
assignee: nobody → Pradeep Kilambi (pkilambi)
Revision history for this message
Pradeep Kilambi (pkilambi) wrote :
Changed in tripleo:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/mitaka)

Reviewed: https://review.openstack.org/344823
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=e1886c1385a4d3c9147dc02cf15b84509661a01a
Submitter: Jenkins
Branch: stable/mitaka

commit e1886c1385a4d3c9147dc02cf15b84509661a01a
Author: Pradeep Kilambi <email address hidden>
Date: Wed Jul 20 09:04:53 2016 -0400

    Add base constraint so gnocchi metricd is tied to core-clone

    Currently if we manually restart the cluster saometimes gnocchi statsd
    doesnt comeup as galera is not up yet. This should tie the metrics to
    core and follow the order.

    Closes-bug: #1604860

    Change-Id: I37117c1d8fda621c4ccb975b5074d5e5044c3b0f

tags: added: in-stable-mitaka
Changed in tripleo:
status: In Progress → Fix Committed
Changed in tripleo:
milestone: none → newton-3
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/tripleo-heat-templates 2.1.0

This issue was fixed in the openstack/tripleo-heat-templates 2.1.0 release.

Changed in tripleo:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 5.0.0.0b3

This issue was fixed in the openstack/puppet-tripleo 5.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 2.1.0

This issue was fixed in the openstack/tripleo-heat-templates 2.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.