KSC's TCPKeepAliveAdapter should specify keepidle to prevent connections from hanging for hours

Bug #1477275 reported by Ian Cordasco
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
keystoneauth
Fix Released
Undecided
Ian Cordasco
python-keystoneclient
Fix Released
Medium
Ian Cordasco
python-novaclient
Fix Released
Undecided
Ian Cordasco

Bug Description

Currently, infra's nodepool is suffering from connections hanging for hours at a time because keepidle is not specified as part of the TCPKeepAliveAdapter in KSC.

Ian Cordasco (icordasc)
Changed in python-keystoneclient:
assignee: nobody → Ian Cordasco (icordasc)
Changed in python-novaclient:
assignee: nobody → Ian Cordasco (icordasc)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-keystoneclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/204741

Changed in python-keystoneclient:
status: New → In Progress
Dolph Mathews (dolph)
Changed in python-keystoneclient:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-novaclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/204745

Changed in python-novaclient:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystoneauth (master)

Fix proposed to branch: master
Review: https://review.openstack.org/205276

Changed in keystoneauth:
assignee: nobody → Ian Cordasco (icordasc)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-keystoneclient (master)

Reviewed: https://review.openstack.org/204741
Committed: https://git.openstack.org/cgit/openstack/python-keystoneclient/commit/?id=c6b14f94c5021452796d7bd151c2c98ae983afdd
Submitter: Jenkins
Branch: master

commit c6b14f94c5021452796d7bd151c2c98ae983afdd
Author: Ian Cordasco <email address hidden>
Date: Wed Jul 22 14:53:32 2015 -0500

    Set reasonable defaults for TCP Keep-Alive

    Previously we simply turned on TCP Keep-Alive which relied on
    per-distribution, per-operating system defaults for keep-alive options.
    Here we set reasonable defaults since long running processes can get
    stuck for hours on end by using system defaults. This also adds comments
    around the options to explain why they're being set.

    Closes-bug: 1477275
    Related-bug: 1323862
    Change-Id: Ibd53ae2d4d2455db0ebc9951e5c764befc57850f

Changed in python-keystoneclient:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to python-keystoneclient (feature/keystoneauth_integration)

Fix proposed to branch: feature/keystoneauth_integration
Review: https://review.openstack.org/207267

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-novaclient (master)

Reviewed: https://review.openstack.org/204745
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=c69c38c58f6789cdaa89d51b696efc0c7b5f825d
Submitter: Jenkins
Branch: master

commit c69c38c58f6789cdaa89d51b696efc0c7b5f825d
Author: Ian Cordasco <email address hidden>
Date: Wed Jul 22 15:13:01 2015 -0500

    Use keystoneclient's TCPKeepAliveAdapter

    When novaclient isn't using a session from keystoneclient, it needs to
    set reasonable TCP Keep-Alive values otherwise the operating system
    defaults may cause the client to hang for hours before a connection will
    time out. Using keystoneclient's adpater (which sets good defaults) will
    allow us to not have to maintain this adapter here and to benefit from
    their defaults.

    Closes-bug: 1477275
    Related-bug: 1323862
    Depends-On: Ibd53ae2d4d2455db0ebc9951e5c764befc57850f
    Change-Id: I1924bd96eb1a4bac5d57a5cc5d5461acb3f7f5ac

Changed in python-novaclient:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to keystoneauth (master)

Reviewed: https://review.openstack.org/205276
Committed: https://git.openstack.org/cgit/openstack/keystoneauth/commit/?id=4b095414614e5e1c964f872614285bf4d77e11e9
Submitter: Jenkins
Branch: master

commit 4b095414614e5e1c964f872614285bf4d77e11e9
Author: Ian Cordasco <email address hidden>
Date: Thu Jul 23 15:18:05 2015 -0500

    Set reasonable defaults for TCP Keep-Alive

    Previously we simply turned on TCP Keep-Alive which relied on
    per-distribution, per-operating system defaults for keep-alive options.
    Here we set reasonable defaults since long running processes can get
    stuck for hours on end by using system defaults. This also adds comments
    around the options to explain why they're being set.

    Closes-bug: 1477275
    Related-bug: 1323862
    Change-Id: Id7d5cb4d5c7012514359e33ce58992cc0830c4e8

Changed in keystoneauth:
status: In Progress → Fix Committed
Changed in keystoneauth:
milestone: none → 0.4.0
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-keystoneclient (feature/keystoneauth_integration)
Download full text (22.7 KiB)

Reviewed: https://review.openstack.org/207267
Committed: https://git.openstack.org/cgit/openstack/python-keystoneclient/commit/?id=4e498a54d0034b2ce5c87130f080ff580d241600
Submitter: Jenkins
Branch: feature/keystoneauth_integration

commit 51d9d123a8df79f6b84d196d41f1008f8d6033d4
Author: Brant Knudson <email address hidden>
Date: Wed Aug 5 12:28:30 2015 -0500

    Deprecate openstack.common.apiclient

    Deprecate the apiclient from oslo-incubator so we can get rid of
    it.

    bp deprecations

    Change-Id: I1c761933816da03b6c625f14d0aac43f206e88d7

commit 16e834dd4597314d79cf4fb0bb98449e6552f804
Author: Brant Knudson <email address hidden>
Date: Wed Aug 5 11:17:34 2015 -0500

    Move apiclient.base.Resource into keystoneclient

    keystoneclient is using apiclient.base and in order to properly
    deprecate and eventually get rid of apiclient we need to move the
    symbols that keystoneclient uses out of apiclient.

    This change moves apiclient.base.Resource into keystoneclient.base
    by merging apiclient.base.Resource into the existing
    keystoneclient.base.Resource. apiclient.base.Resource is now
    renaming keystoneclient.base.Resource for backwards-compatibility.

    Change-Id: Id479711b7c9437aaf171def6976aab8b303ec56d

commit 26534dadb1d0be00b87b632a038839ab1c18cfe4
Author: Brant Knudson <email address hidden>
Date: Tue Aug 4 19:57:26 2015 -0500

    oslo-incubator apiclient.exceptions to keystoneclient.exceptions

    Applications are using exceptions out of
    keystoneclient.openstack.common.apiclient.exceptions so it's part
    of the public interface. But since it's from oslo-incubator it's
    not considered stable. Since keystoneclient is all stable this
    creates bad situation.

    With this change, all the symbols out of apiclient.exceptions are
    moved into keystoneclient.exceptions rather than the other way
    around (keystoneclient.exceptions used to re-export all of
    apiclient.exceptions). Now we're in control of the
    apiclient.exceptions and keystoneclient.exceptions that
    applications are using.

    Closes-Bug: 1481806
    Change-Id: Ib3afa86b9d276f6a45d1ecd6f9157ee02ec8570d

commit eaa7ddd7443ca166f6646e808dcad959811d158b
Author: Brant Knudson <email address hidden>
Date: Sun Jul 26 06:53:58 2015 -0500

    Proper deprecation for HTTPClient session and adapter properties

    HTTPClient's forwarded session and adapter properties weren't
    properly deprecated since the deprecations was only mentioned in
    the docstring. Proper deprecation requires use of warnings/
    debtcollector and documentation.

    bp deprecations

    Change-Id: Iea76d7bbc3bdeb13f7fdb097f13e007b4dd85c8d

commit 0c2fef51d2b90df088d30e9b6c5079caae7c6578
Author: Brant Knudson <email address hidden>
Date: Fri Jul 24 15:52:57 2015 -0500

    Proper deprecation for HTTPClient.request methods

    HTTPClient.request and related methods weren't properly
    deprecated since they were only mentioned in the docstrings.
    Proper deprecation requires use of warnings/debtcollector and
    documentation.

    Also, fixed places where the deprecated request m...

Changed in python-novaclient:
milestone: none → 2.27.0
status: Fix Committed → Fix Released
Changed in python-keystoneclient:
milestone: none → 1.7.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.