OpenDaylight HA deployment fails to launch instance due to Port Status failures

Bug #1718508 reported by Tim Rozet
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
puppet-neutron
Fix Released
Undecided
Tim Rozet
tripleo
Fix Released
Medium
Tim Rozet

Bug Description

Introduced in OpenDaylight is the ability for ODL to report to Neutron when the VIF has been plugged in OVS and report proper port state to Neutron. However, the current implementation does not work in an HA environment. Only a single OpenDaylight in a cluster will open a websocket connection to Neutron and port status updates will not be sent if the leader of the cluster is not the same one that has initiated the websocket connection. Therefore we need to disable port status feature by changing ML2 conf.

Revision history for this message
Tim Rozet (trozet) wrote :

Impact of this is no instances can be brought up at all post deployment, so this is a blocker.

Changed in tripleo:
status: New → In Progress
assignee: nobody → Tim Rozet (trozet)
Changed in puppet-neutron:
assignee: nobody → Tim Rozet (trozet)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/505781

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (master)

Fix proposed to branch: master
Review: https://review.openstack.org/505788

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-neutron (master)

Reviewed: https://review.openstack.org/505781
Committed: https://git.openstack.org/cgit/openstack/puppet-neutron/commit/?id=2417be6fae4df4003be0471a405d09a653d29f5d
Submitter: Jenkins
Branch: master

commit 2417be6fae4df4003be0471a405d09a653d29f5d
Author: Tim Rozet <email address hidden>
Date: Wed Sep 20 15:04:21 2017 -0400

    Adds the ability to specify ML2 ODL Features

    This is required in order to override the negotiated default features
    with OpenDaylight. More specifically this is required so that we can
    override the features list not to include operational-port-status to
    fix the following bug.

    Partial-Bug: 1718508

    Change-Id: I7eb752ad692e5522051f8393376890fcac9a09fe
    Signed-off-by: Tim Rozet <email address hidden>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-neutron (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/506783

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-neutron (stable/pike)

Reviewed: https://review.openstack.org/506783
Committed: https://git.openstack.org/cgit/openstack/puppet-neutron/commit/?id=889da598ecaa191c183cfb543d4f2f1bcab6b920
Submitter: Jenkins
Branch: stable/pike

commit 889da598ecaa191c183cfb543d4f2f1bcab6b920
Author: Tim Rozet <email address hidden>
Date: Wed Sep 20 15:04:21 2017 -0400

    Adds the ability to specify ML2 ODL Features

    This is required in order to override the negotiated default features
    with OpenDaylight. More specifically this is required so that we can
    override the features list not to include operational-port-status to
    fix the following bug.

    Partial-Bug: 1718508

    Change-Id: I7eb752ad692e5522051f8393376890fcac9a09fe
    Signed-off-by: Tim Rozet <email address hidden>
    (cherry picked from commit 2417be6fae4df4003be0471a405d09a653d29f5d)

tags: added: in-stable-pike
Changed in tripleo:
importance: Undecided → Medium
milestone: none → queens-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/505788
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=228d7b456c6d5c4958c9add8f9021a83a4360510
Submitter: Jenkins
Branch: master

commit 228d7b456c6d5c4958c9add8f9021a83a4360510
Author: Tim Rozet <email address hidden>
Date: Wed Sep 20 15:49:50 2017 -0400

    Disables port status updates with ODL in HA

    ODL enables a feature by default to communicate port state to Neutron
    via a websocket connection. The current implementation does not work in
    HA, but does work with a noHA deployment. Therefore this patch disables
    port status for HA deployments only until there is proper support.

    Depends-On: I7eb752ad692e5522051f8393376890fcac9a09fe

    Closes-Bug: 1718508

    Change-Id: I13b5b72285d3c70cdee4d81678470d52be385aaf
    Signed-off-by: Tim Rozet <email address hidden>

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/508034

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (stable/pike)

Reviewed: https://review.openstack.org/508034
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=67e1e603a6b81e7532eb2cac2e27eec1ea413d32
Submitter: Jenkins
Branch: stable/pike

commit 67e1e603a6b81e7532eb2cac2e27eec1ea413d32
Author: Tim Rozet <email address hidden>
Date: Wed Sep 20 15:49:50 2017 -0400

    Disables port status updates with ODL in HA

    ODL enables a feature by default to communicate port state to Neutron
    via a websocket connection. The current implementation does not work in
    HA, but does work with a noHA deployment. Therefore this patch disables
    port status for HA deployments only until there is proper support.

    Depends-On: I7eb752ad692e5522051f8393376890fcac9a09fe

    Closes-Bug: 1718508

    Change-Id: I13b5b72285d3c70cdee4d81678470d52be385aaf
    Signed-off-by: Tim Rozet <email address hidden>
    (cherry picked from commit 228d7b456c6d5c4958c9add8f9021a83a4360510)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/509834

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/509834
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=2471a8669d35f8b35ed00e9365623b37e335cd79
Submitter: Jenkins
Branch: master

commit 2471a8669d35f8b35ed00e9365623b37e335cd79
Author: Tim Rozet <email address hidden>
Date: Thu Oct 5 12:42:52 2017 -0400

    Disables port status for all ODL deployments

    Port status was already disabled in HA deployments pending a fix for:
    https://bugs.opendaylight.org//show_bug.cgi?id=9147

    However even in noha deployments port status will not work because ODL
    is unable to bind to a specific IP for websocket, meaning it binds to
    all IPs and haproxy cannot bind the VIP. Therefore we need to disable
    it for all deployments until also this bug is fixed:
    https://bugs.opendaylight.org//show_bug.cgi?id=9256

    Related-Bug: 1718508

    Change-Id: I2f2dc3ece97c97fc8477d4129d69719866a7f0c1
    Signed-off-by: Tim Rozet <email address hidden>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to puppet-tripleo (stable/pike)

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/510878

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 7.4.2

This issue was fixed in the openstack/puppet-tripleo 7.4.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (stable/pike)

Reviewed: https://review.openstack.org/510878
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=22ff1cf840107663f3aeae5b870a2d4615094d14
Submitter: Jenkins
Branch: stable/pike

commit 22ff1cf840107663f3aeae5b870a2d4615094d14
Author: Tim Rozet <email address hidden>
Date: Thu Oct 5 12:42:52 2017 -0400

    Disables port status for all ODL deployments

    Port status was already disabled in HA deployments pending a fix for:
    https://bugs.opendaylight.org//show_bug.cgi?id=9147

    However even in noha deployments port status will not work because ODL
    is unable to bind to a specific IP for websocket, meaning it binds to
    all IPs and haproxy cannot bind the VIP. Therefore we need to disable
    it for all deployments until also this bug is fixed:
    https://bugs.opendaylight.org//show_bug.cgi?id=9256

    Related-Bug: 1718508

    Change-Id: I2f2dc3ece97c97fc8477d4129d69719866a7f0c1
    Signed-off-by: Tim Rozet <email address hidden>
    (cherry picked from commit 2471a8669d35f8b35ed00e9365623b37e335cd79)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 8.0.0

This issue was fixed in the openstack/puppet-tripleo 8.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/515576
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=bc3feec75f8dc1bac6691ce8cd05a37c27aa5e42
Submitter: Zuul
Branch: master

commit bc3feec75f8dc1bac6691ce8cd05a37c27aa5e42
Author: Tim Rozet <email address hidden>
Date: Thu Oct 26 22:06:50 2017 -0500

    Enables websocket based port status for OpenDaylight

    Port Status in ODL was previously disabled due to bugs which have now
    been resolved. This patch enables it via Neutron ML2 conf. Port status
    is communicated by ODL through a websocket connection on port 8185.
    Therefore we need to enable haproxy to load balance across that port.

    Related-Bug: 1718508

    Change-Id: Iebdcc6404f5503eeb45b39380b3f198be175514b
    Signed-off-by: Tim Rozet <email address hidden>

Changed in puppet-neutron:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.