keystone error on evaluator startup

Bug #1518447 reported by gordon chung
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Aodh
Fix Released
Medium
ZhiQiang Fan
Liberty
Fix Released
Undecided
ZhiQiang Fan

Bug Description

http://logs.openstack.org/23/242423/2/check/gate-ceilometer-dsvm-integration/755caa6/logs/screen-aodh-evaluator.txt.gz

2015-11-18 16:53:02.820 14630 ERROR stevedore.extension [-] Could not load 'threshold': Unable to establish connection to http://127.0.0.1:5000/
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension [-] Unable to establish connection to http://127.0.0.1:5000/
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension Traceback (most recent call last):
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/stevedore/extension.py", line 162, in _load_plugins
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension verify_requirements,
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/stevedore/extension.py", line 187, in _load_one_plugin
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension obj = plugin(*invoke_args, **invoke_kwds)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/opt/stack/new/aodh/aodh/evaluator/threshold.py", line 63, in __init__
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension os_project_domain_id=auth_config.os_project_domain_id,
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/client.py", line 417, in get_client
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return Client(version, endpoint, **kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/client.py", line 364, in Client
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return client_class(*args, **client_kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/v2/client.py", line 65, in __init__
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension self.alarm_client, aodh_enabled = self._get_alarm_client(**kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/v2/client.py", line 113, in _get_alarm_client
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension kwargs.get('timeout'))
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/client.py", line 274, in redirect_to_aodh_endpoint
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension ks_session = _get_keystone_session(**ks_kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/client.py", line 88, in _get_keystone_session
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension v2_auth_url, v3_auth_url = _discover_auth_versions(ks_session, auth_url)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/ceilometerclient/client.py", line 38, in _discover_auth_versions
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension ks_discover = discover.Discover(session=session, auth_url=auth_url)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return func(*args, **kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/discover.py", line 181, in __init__
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension authenticated=authenticated)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return func(*args, **kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/_discover.py", line 142, in __init__
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension authenticated=authenticated)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return func(*args, **kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/_discover.py", line 40, in get_version_data
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension resp = session.get(url, headers=headers, authenticated=authenticated)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 494, in get
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return self.request(url, 'GET', **kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/utils.py", line 337, in inner
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension return func(*args, **kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 387, in request
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension resp = send(**kwargs)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension File "/usr/local/lib/python2.7/dist-packages/keystoneclient/session.py", line 431, in _send_request
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension raise exceptions.ConnectionRefused(msg)
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension ConnectionRefused: Unable to establish connection to http://127.0.0.1:5000/
2015-11-18 16:53:02.820 14630 ERROR stevedore.extension

ZhiQiang Fan (aji-zqfan)
Changed in aodh:
assignee: nobody → ZhiQiang Fan (aji-zqfan)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to aodh (master)

Fix proposed to branch: master
Review: https://review.openstack.org/248651

Changed in aodh:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to aodh (master)

Reviewed: https://review.openstack.org/248651
Committed: https://git.openstack.org/cgit/openstack/aodh/commit/?id=3592c67acea6b7f3a0e5c070bd940fdfb0a5db4f
Submitter: Jenkins
Branch: master

commit 3592c67acea6b7f3a0e5c070bd940fdfb0a5db4f
Author: ZhiQiang Fan <email address hidden>
Date: Tue Nov 24 04:04:02 2015 +0800

    initialize ceilometerclient when we use it

    There is a chance that ceilometerclient cannot be initialized if
    keystone is not available, putting the initialization in __init__
    will fail to load threshold extension for such case.

    This patch moves the initialization out from __init__ to a dedicate
    property method.

    Change-Id: I24895126c3efe9920390401e4d10a2ce6b1f1376
    Closes-Bug: #1518447

Changed in aodh:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to aodh (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/250339

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to aodh (stable/liberty)

Reviewed: https://review.openstack.org/250339
Committed: https://git.openstack.org/cgit/openstack/aodh/commit/?id=c88f02f83dfd5dadbf5c90f13a0a34157105b8ee
Submitter: Jenkins
Branch: stable/liberty

commit c88f02f83dfd5dadbf5c90f13a0a34157105b8ee
Author: ZhiQiang Fan <email address hidden>
Date: Tue Nov 24 04:04:02 2015 +0800

    initialize ceilometerclient when we use it

    There is a chance that ceilometerclient cannot be initialized if
    keystone is not available, putting the initialization in __init__
    will fail to load threshold extension for such case.

    This patch moves the initialization out from __init__ to a dedicate
    property method.

    Conflicts:
     aodh/evaluator/threshold.py

    Change-Id: I24895126c3efe9920390401e4d10a2ce6b1f1376
    Closes-Bug: #1518447
    (cherry picked from commit 3592c67acea6b7f3a0e5c070bd940fdfb0a5db4f)

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/aodh 2.0.0.0b1

This issue was fixed in the openstack/aodh 2.0.0.0b1 development milestone.

Changed in aodh:
status: Fix Committed → Fix Released
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/aodh 1.1.1

This issue was fixed in the openstack/aodh 1.1.1 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.