gnocchi: dispatcher stucks when gnocchi API is restarted

Bug #1498823 reported by Mehdi Abaakouk
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Critical
Mehdi Abaakouk

Bug Description

Hi,

gnocchi dispatcher stucks when gnocchi API is restarted or if collector received samples before gnocchi API is started

This is due to https://github.com/kennethreitz/requests/issues/2784

Cheers,

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/226700

Changed in ceilometer:
assignee: nobody → Mehdi Abaakouk (sileht)
status: New → In Progress
gordon chung (chungg)
Changed in ceilometer:
milestone: none → liberty-rc1
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)

Reviewed: https://review.openstack.org/226700
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=03769b699f6f98e775c2b8deacee8cd45cbc7095
Submitter: Jenkins
Branch: master

commit 03769b699f6f98e775c2b8deacee8cd45cbc7095
Author: Mehdi Abaakouk <email address hidden>
Date: Wed Sep 23 13:05:05 2015 +0200

    Workaround requests/urllib connection leaks

    When a ConnectionError occurs, python-requests <= 2.8.0
    and/or urllib3 <= 1.11, connections are not returned to
    the pool.

    This change adds a CustomHTTPAdapter to workaround this
    issue.

    This HTTPAdapter doesn't trigger theses urllib3 issues.
    This HTTPAdapter workaround this by enforcing preloading
    of the response. When enabled, urllib3 releases the connection
    to the pool immediately after its usage, and doesn't trigger the
    issue.

    By enforcing preloading, this break some requests
    features (like stream) that we didn't use into our GnocchiClient

    Upstream bugs:

    * https://github.com/shazow/urllib3/issues/659
    * https://github.com/shazow/urllib3/issues/651
    * https://github.com/shazow/urllib3/issues/644
    * https://github.com/kennethreitz/requests/issues/2687

    We could remove this when requests 2.8.0 will be released

    Closes-bug: #1498823
    Change-Id: I7d40ade927b834909e230613777cba1f7537c0ec

Changed in ceilometer:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ceilometer:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ceilometer:
milestone: liberty-rc1 → 5.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.