neutron-lbaas health monitor timeout and delay values interpreted as milliseconds

Bug #1600326 reported by Dustin Lundquist
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Paul Glass
octavia
Fix Released
Medium
Paul Glass

Bug Description

The timeout and delay values on the health monitor objects in Neutron LBaaS are purportedly in units of seconds, but the numeric value is passed all the the way down to the HAProxy configuration[1] file (in both the HAProxy namespace driver and Octavia) where it is interpreted in milliseconds:

* https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#4.2-timeout%20check
* https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.2-inter

Due to this unit mismatch, a user may configure a pool with a reasonable 10 second timeout, and the service may appear to function normally until even a small load causes the backend servers to exceed a 10 millisecond timeout and then they are removed from the pool.

A timeout value less than one second is useful some settings, such as monitoring a pool of backend servers serving static content, let the database field stores this value as an integer.

1: https://github.com/openstack/neutron-lbaas/blob/b322615e4869eb42ed7888a3492eae4a52f3b4db/neutron_lbaas/services/loadbalancer/drivers/haproxy/templates/haproxy_proxies.j2#L72

description: updated
summary: - neutron-lbaas health monitor timeout and delay values intepreated as
+ neutron-lbaas health monitor timeout and delay values interpreted as
milliseconds
Changed in neutron:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Dustin Lundquist (dlundquist) wrote :

Updating template to use value as seconds was discussed in https://review.openstack.org/#/c/339260/, but favored updating documentation instead so not to alter behavior of existing deployed load balancers.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/heat 7.0.0.0b2

This issue was fixed in the openstack/heat 7.0.0.0b2 development milestone.

Changed in octavia:
importance: Undecided → Medium
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/python-neutronclient 5.0.0

This issue was fixed in the openstack/python-neutronclient 5.0.0 release.

Zhang Ruochen (roykingz)
information type: Public → Public Security
Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

Is there a reason why this is reported as a security issue ?

information type: Public Security → Public
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron-lbaas (master)

Fix proposed to branch: master
Review: https://review.openstack.org/380658

Changed in neutron:
assignee: nobody → Paul Glass (pnglass)
status: Confirmed → In Progress
Changed in octavia:
assignee: nobody → Paul Glass (pnglass)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (master)

Fix proposed to branch: master
Review: https://review.openstack.org/380660

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (master)

Reviewed: https://review.openstack.org/380660
Committed: https://git.openstack.org/cgit/openstack/octavia/commit/?id=ef11747a56ed210d9ae59e7c08b08fb6b6dc604b
Submitter: Jenkins
Branch: master

commit ef11747a56ed210d9ae59e7c08b08fb6b6dc604b
Author: Paul Glass <email address hidden>
Date: Fri Sep 30 21:39:52 2016 +0000

    Switch HAProxy health check timeout to seconds

    Change-Id: If8166b8e76ca6c1b15963ef99bec07ff2a6fb118
    Closes-Bug: #1600326

Changed in octavia:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron-lbaas (master)

Reviewed: https://review.openstack.org/380658
Committed: https://git.openstack.org/cgit/openstack/neutron-lbaas/commit/?id=75faa63ee40a7de8eaa7c8f2b2f95038c0e92158
Submitter: Jenkins
Branch: master

commit 75faa63ee40a7de8eaa7c8f2b2f95038c0e92158
Author: Paul Glass <email address hidden>
Date: Fri Sep 30 21:35:56 2016 +0000

    Switch HAProxy health check timeout to seconds

    Change-Id: I6e3c68be26cdf9ed3e09b7f0bd92a52491b5e866
    Closes-Bug: #1600326

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron-lbaas 10.0.0.0b1

This issue was fixed in the openstack/neutron-lbaas 10.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron-lbaas (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/409917

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron-lbaas (stable/newton)

Reviewed: https://review.openstack.org/409917
Committed: https://git.openstack.org/cgit/openstack/neutron-lbaas/commit/?id=ebae55c633d8bb0fd53d4c2dd8999893ce17f953
Submitter: Jenkins
Branch: stable/newton

commit ebae55c633d8bb0fd53d4c2dd8999893ce17f953
Author: Paul Glass <email address hidden>
Date: Fri Sep 30 21:35:56 2016 +0000

    Switch HAProxy health check timeout to seconds

    Change-Id: I6e3c68be26cdf9ed3e09b7f0bd92a52491b5e866
    Closes-Bug: #1600326
    (cherry picked from commit 75faa63ee40a7de8eaa7c8f2b2f95038c0e92158)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 0.10.0

This issue was fixed in the openstack/octavia 0.10.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/451564

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/newton)

Reviewed: https://review.openstack.org/451564
Committed: https://git.openstack.org/cgit/openstack/octavia/commit/?id=bb79b3df3333ac4d7778753e343b1ae9631227a7
Submitter: Jenkins
Branch: stable/newton

commit bb79b3df3333ac4d7778753e343b1ae9631227a7
Author: Paul Glass <email address hidden>
Date: Fri Sep 30 21:39:52 2016 +0000

    Switch HAProxy health check timeout to seconds

    Change-Id: If8166b8e76ca6c1b15963ef99bec07ff2a6fb118
    Closes-Bug: #1600326
    (cherry picked from commit ef11747a56ed210d9ae59e7c08b08fb6b6dc604b)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron-lbaas 9.2.1

This issue was fixed in the openstack/neutron-lbaas 9.2.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 0.9.2

This issue was fixed in the openstack/octavia 0.9.2 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.