[RFE] Distributed Portbinding for all port types
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| neutron |
Wishlist
|
Miguel Lavalle |
Bug Description
Summary
=======
Today only DVR ports can be bound to multiple hosts. But having a port bound to
multiple hosts does also make sense for a compute port during live migration.
For a certain period of time the port could be bound to the source and
target at the same time (Although only one is being used). The information of both bindings needs to be set and accessible from Nova via the ReST API.
Use Cases
=========
* Instance in error state when portbinding fails
In the live migration process, port binding is triggered by Nova after the migration already succeeded. If port binding fails, the instance is stuck in error state. If portbinding for the target node would be done in pre_live_migration, migration could be aborted on a binding failure and the instance would still be active on the migration source host. But we cannot just do so, as some TOR mech drivers would shut down the source port after the binding has been updated, although the instance is still active on the source. If we could bind a compute port to both hosts, such drivers could keep the source port open, and already process the target port in parallel.
* Live Migration between hosts running different l2 agents
Another use case is live migration between hosts that run different l2 agents. This requires that Nova updates the instance definition before migration is executed (in case of libvirt, update the domain.xml with target interface definition).
A specialized variant of this use case is the migration from an agent with one firewall driver to another (e.g. from ovs hybrid-fw driver to new ovs conntrackd firewall driver).
* Live migration with MacVTap agent when different physnet mappings is used
The third use case is live migration with MacVTap agent. Today it has some restrictions with live migration in some special scenarios [1]. It requires an update on the instance definition (libvirt domain.xml) before the migration started.
For updating the definition in time, a portbinding for the migration target node is required even before the migration started. Along the argumentation above, we need a compute port bound to multiple hosts.
Proposed Change
===============
* A refactoring of the database is required to make a normal port a special case of a distributed port. This was planned since a long time but was never finished. The efforts are tracked via this bug [1]. The patches still need to be rebased to get that going again.
* ReST API changes are required to externalize the bindings. To not overload the port API, a new subresource "bindings" could be created (like /ports/
Nova Changes
============
* In pre_live_migration, nova would add a new binding for the migration target host to the port - this triggers portbinding in Neutron.
* Before migration starts, Nova would access the binding information for the target host. It would abort on "binding_failed" vif type. Otherwise it would modify the instance definition (e.g. domain.xml) for the migration target with this binding information.
* After live migration succeeded, Nova would remove the original port_binding. On Rollback, it would just remove the target port_binding.
Those changes are tracked via the following Nova blueprint [4]
Open Questions
==============
* This RFE is based on bug [1]. How to track those dependencies? Or should the content of this bug become part of this effort?
* Similar with the macvtap live migration bug [3]
* How does this effort correlate to the the RFE for externalizing multi-segment networks [2]?
[1] https:/
[2] https:/
[3] https:/
[4] https:/
summary: |
- [RFE] Distributed Portbinding for all ports + [RFE] Distributed Portbinding for all port tpyes |
description: | updated |
description: | updated |
Changed in neutron: | |
importance: | Undecided → Wishlist |
summary: |
- [RFE] Distributed Portbinding for all port tpyes + [RFE] Distributed Portbinding for all port types |
Thanks Armando, any feedback is highly welcome!
venkata anil (anil-venkata) wrote : | #3 |
Can we refer this port as "migrated" port rather than "distributed" port?
We already have DVR and HA router ports(DEVICE_
What about "distributed portbinding for migrated ports"? Cause basically I want the functionality that is available for DVR today also for compute ports during migration + some API.
Changed in neutron: | |
assignee: | nobody → Andreas Scheuring (andreas-scheuring) |
status: | Confirmed → In Progress |
Changed in neutron: | |
status: | In Progress → Confirmed |
Changed in neutron: | |
status: | Confirmed → Triaged |
I promise this is next on my stack. I have never had enough time to go over this one in peace.
I reviewed the spec, I think this needs to mature a little longer, but a reduction in scope seems in order.
This needs more scoping. Let's iterate on the spec to gather a better understanding of the beast at play.
Fix proposed to branch: master
Review: https:/
Changed in neutron: | |
status: | Triaged → In Progress |
Changed in neutron: | |
status: | In Progress → Confirmed |
Changed in neutron: | |
status: | Confirmed → Triaged |
Changed in neutron: | |
status: | Triaged → In Progress |
Changed in neutron: | |
status: | In Progress → Triaged |
tags: |
added: rfe-postponed removed: rfe |
Let's iterate on the spec until we can solidify a plan, this is now Ocata material, though some prep work will be evaluated for Newton on a best effort basis.
Changed in neutron: | |
status: | Triaged → In Progress |
Changed in neutron: | |
status: | In Progress → Triaged |
Changed in neutron: | |
status: | Triaged → In Progress |
Change abandoned by Andreas Scheuring (<email address hidden>) on branch: master
Review: https:/
Changed in neutron: | |
assignee: | Andreas Scheuring (andreas-scheuring) → Brian Stajkowski (brian-stajkowski) |
Fix proposed to branch: master
Review: https:/
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Andreas Scheuring (andreas-scheuring) |
Changed in neutron: | |
assignee: | Andreas Scheuring (andreas-scheuring) → Artur Korzeniewski (artur-korzeniewski) |
Changed in neutron: | |
assignee: | Artur Korzeniewski (artur-korzeniewski) → Anindita Das (anindita-das) |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Changed in neutron: | |
milestone: | none → ocata-2 |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Fix proposed to branch: master
Review: https:/
Changed in neutron: | |
milestone: | ocata-2 → ocata-3 |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Brian Stajkowski (brian-stajkowski) |
Changed in neutron: | |
assignee: | Brian Stajkowski (brian-stajkowski) → Anindita Das (anindita-das) |
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 19eb264aaf4dd7d
Author: Andreas Scheuring <email address hidden>
Date: Fri Apr 22 15:49:45 2016 +0200
Spec for providing Nova portbinding information for live migration
The goal is to provide nova the required portbinding information
for live migration. This spec proposes to allow a port to have
a second inactive binding. During migration, the host that
runs the instance has the active binding, while the binding
belonging to the other host is inactive.
A new API is required to externalize this new inactive binding.
Partial-Bug: #1580880
Co-authored-by: Brian Stajkowski <email address hidden>
Change-Id: I12d1310b8df30f
Changed in neutron: | |
milestone: | ocata-3 → ocata-rc1 |
milestone: | ocata-rc1 → pike-1 |
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 7872508f9f25ea6
Author: Anindita Das <email address hidden>
Date: Tue Nov 29 07:45:22 2016 -0800
[Live Migration] Extend ml2_port_binding table
Extended the Primary Key of ml2_port_binding table to include "hosts"
and added a new column named "status" to indicate the status of the
port similar to the ml2_distributed
PortBinding object with the above changes.
Change-Id: Ie01f83b01ab673
Co-Authored-By: Brian Stajkowski <email address hidden>
Partial-Bug: #1580880
tags: | added: neutron-proactive-backport-potential |
Hi Ihar, I'm not sure if considering this for backports makes sense. It's a large feature rather than a bugfix - and it won't make ocata. But maybe I misunderstand the backport-
tags: | removed: neutron-proactive-backport-potential |
Fix proposed to branch: master
Review: https:/
Changed in neutron: | |
assignee: | Anindita Das (anindita-das) → Jakub Libosvar (libosvar) |
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 26f039390c64a69
Author: Andreas Scheuring <email address hidden>
Date: Tue Apr 25 13:14:13 2017 +0200
Repropose: Spec for providing Nova portbinding information for live migration
This patch is reproposal of change proposed by Andreas at
I12d1310b8d
The goal is to provide nova the required portbinding information
for live migration. This spec proposes to allow a port to have
a second inactive binding. During migration, the host that
runs the instance has the active binding, while the binding
belonging to the other host is inactive.
A new API is required to externalize this new inactive binding.
Partial-Bug: #1580880
Co-authored-by: Brian Stajkowski <email address hidden>
Change-Id: I707e48f6c13f80
Changed in neutron: | |
milestone: | pike-1 → pike-2 |
tags: |
added: rfe-approved removed: rfe-postponed |
Fix proposed to branch: master
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 6803dc6dddb7a68
Author: Jakub Libosvar <email address hidden>
Date: Mon Jul 17 14:35:14 2017 +0000
Introduce API for port bindings extended
This is a neutron-lib part that contains API resource definition of
extended port bindings.
Partial-Bug: #1580880
Change-Id: I431be79975f6fc
Changed in neutron: | |
milestone: | pike-2 → queens-2 |
Changed in neutron: | |
assignee: | Jakub Libosvar (libosvar) → Miguel Lavalle (minsel) |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 4c39634d2853c8a
Author: Miguel Lavalle <email address hidden>
Date: Sun Oct 15 18:21:23 2017 -0500
Add new resource for port bindings
As a result of implementing providing Nova port binding information for
live migration [1], a new resource for callbacks is required. This
resource will be used by the multiple port binding service plugin [2]
to notify the core plugin and other subscribers when create, update and
delete operations are performed on a port binding.
[1] https:/
[2] https:/
Related-Bug: #1580880
Change-Id: Ibbc7af67cf1282
Fix proposed to branch: master
Review: https:/
Fix proposed to branch: master
Review: https:/
Fix proposed to branch: master
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 1d645c8ef30f6a6
Author: Miguel Lavalle <email address hidden>
Date: Wed Apr 25 21:38:46 2018 -0500
Add port bindings to RPC topics resources
This change adds port bindings to the RPC topics resources. It also
adds the 'activate' and 'deactivate' messages for this resource. This
is done in support of the implementation of multiple port bindings.
Change-Id: I5ed542b7a13733
Partial-Bug: #1580880
Changed in neutron: | |
assignee: | Miguel Lavalle (minsel) → Matt Riedemann (mriedem) |
Changed in neutron: | |
assignee: | Matt Riedemann (mriedem) → Miguel Lavalle (minsel) |
Changed in neutron: | |
assignee: | Miguel Lavalle (minsel) → Matt Riedemann (mriedem) |
Changed in neutron: | |
assignee: | Matt Riedemann (mriedem) → Miguel Lavalle (minsel) |
Fix proposed to branch: master
Review: https:/
Fix proposed to branch: master
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit f7b62a7f29e6b88
Author: Jakub Libosvar <email address hidden>
Date: Thu Jul 27 14:15:11 2017 +0000
Multiple port binding for ML2
Functionality is added to the ML2 plugin to handle multiple port
bindings
Co-Authored-By: Anindita Das <email address hidden>
Co-Authored-By: Miguel Lavalle <email address hidden>
Partial-Bug: #1580880
Change-Id: Ie31d4e27e3f55e
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 390b6a531f7bb58
Author: Miguel Lavalle <email address hidden>
Date: Wed Apr 4 16:39:02 2018 -0500
Update get_port_
With the adoption of multiple port bindings, get_port_
updated to select only the active binding
Change-Id: I57ee16aa970527
Partial-Bug: #1580880
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit f374697760c65d3
Author: Miguel Lavalle <email address hidden>
Date: Mon Apr 23 21:53:43 2018 -0500
Add binding_deactivate method to Linux bridge agent
This commit adds a binding_deactivate method to the Linux bridge agent
to receive messages from the ML2 plugin when a binding is de-activated
for a port. After receiving that message, the agent un-plugs the
corresponding tap interface from the port's network bridge.
To support this, a binding_deactivate method is also added to the agents
notifier. Finally, the activate method in the ML2 plugin is updated to
use the binding_deactivate method in the agents notifier.
Change-Id: I3f4e34766791c4
Partial-Bug: #1580880
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit be7ad302e1f8b9a
Author: Miguel Lavalle <email address hidden>
Date: Mon May 7 16:39:56 2018 -0500
Support binding_deactivate in CacheBackedPlug
CacheBacked
updates to L2 agents. The agents retrieve the resources updates locally
from the cache implemented by it. CacheBackedPlug
server notifications to the agents, such as port_delete or port_update,
based on the updated data received by the cache. This commit adds code
to CacheBackedPlug
from the server to the agents
Change-Id: I023ccbd405bc41
Partial-Bug: #1580880
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 0694bebd6de3934
Author: Miguel Lavalle <email address hidden>
Date: Wed May 9 10:39:05 2018 -0500
Add binding de-activation to OVS agent
To support multiple port bindings, the L2 agents have to have the
capability to handle binding deactivation notifications from the
Neutron server. This patch adds the necessary code to the OVS agent.
After receiving the notification, the agent un-plugs the corresponding
VIF from the integration bridge.
Change-Id: I78178de2039cca
Partial-Bug: #1580880
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 5c3bf124966a310
Author: Miguel Lavalle <email address hidden>
Date: Sat Jun 9 20:03:39 2018 -0500
Add binding activation to OVS agent
As part of the implementation of multiple port bindings [1], add binding
activation support to the OVS agent. This will enable the execution in
OVS agents of the complete sequence of steps outlined in [1] during an
instance migration:
1) Create inactive port bindings for destination host
2) Migrate the instance to the destination host and plug its VIFs
3) Activate the port bindings in the destination host
4) Delete the port bindings for the source host
[1] https:/
Change-Id: Iabca39364ec956
Partial-Bug: #1580880
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit f7064f2b6c6ba1d
Author: Miguel Lavalle <email address hidden>
Date: Sat Jun 23 22:58:42 2018 -0500
Add binding activation to the Linuxbridge agent
As part of the implementation of multiple port bindings [1], add binding
activation support to the linux bridge agent. This will enable the
execution with linux bridge agents of the complete sequence of steps
outlined in [1] during an instance migration:
1) Create inactive port bindings for destination host
2) Migrate the instance to the destination host and plug its VIFs
3) Activate the port bindings in the destination host
4) Delete the port bindings for the source host
[1] https:/
Change-Id: I2c937cc0a551e5
Partial-Bug: #1580880
Fix proposed to branch: master
Review: https:/
Changed in neutron: | |
assignee: | Miguel Lavalle (minsel) → Pawel Suder (pasuder) |
Change abandoned by Paweł Suder (<email address hidden>) on branch: master
Review: https:/
Reason: Thank you Miguel for clarification :))
Changed in neutron: | |
assignee: | Pawel Suder (pasuder) → nobody |
Changed in neutron: | |
assignee: | nobody → Miguel Lavalle (minsel) |
status: | In Progress → Fix Released |
I cannot help but think that we're somewhat abusing the term 'distributed' here, as in the use cases you presented the distributed nature of the binding is only short lived. I'll deep dive in the spec but I sense we may end up creating more trouble that we aim to address.