Pre-created ports get deleted on VM delete

[Deepak Jadiya]

1) Pre create a port using port-create
2) Boot a VM with nova boot --nic port_id=<created port>
3) Delete a VM.

Expected: VM should boot using provided port_id at boot time.
When VM is deleted, port corresponding to pre-created port_id should not get deleted,
as a lot of application, security settings could have port properties configured in them in a large network.

Observed behavior:
There is no way, I could prevent port_id associated with VM from being deleted with nova delete.

[Deepak Jadiya (deepak-jadiya)]

Observe using: quantum port-list -c id -c mac_address -c fixed_ips -c device_owner -c device_id

[dan wendlandt (danwent)]

This is likely a bug in nova.

[dan wendlandt (danwent)]

We've been seeing a fair number of bugs in less commonly used nova networking functionality with quantum (adding fixed_ips, displaying floating ips, and this). I think we should have a BP on this in havana. I'll add it to my list of summit items.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/77043

[Steve Baker (steve-stevebaker)]

Adding heat to this bug since doing a stack update on a stack containing a port and a server will fail if the server is replaced.

[Aaron Rosen (arosen)]

Bumping to high since this also affects heat.

[Clint Byrum (clint-fewbar)]

This affects Heat, but it isn't a bug in Heat. Marking Invalid.

[Steve Baker (steve-stevebaker)]

Could this be marked Critical for Nova? Currently in heat doing a stack-update in the following scenario fails 100% of the time because the port disappears:
- a template containing a server resource and a port resource
- a stack-update resulting in the server resource being replaced

This affects every template which has a floating IP resource, which requires a port

[Steve Baker (steve-stevebaker)]

Interestingly, if the port is updated to clear device_owner and device_id, *and* set admin_state_up=False then nova doesn't delete the port on server delete or nova interface-detach

[Sean Dague (sdague)]

The proposed patch has been in merge conflict for months.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/121693

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/121693
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=30ece56a2118bff073e36a108324d12ee646fab6
Submitter: Jenkins
Branch: master

commit 30ece56a2118bff073e36a108324d12ee646fab6
Author: Steve Baker <email address hidden>
Date: Mon Sep 15 16:37:29 2014 +1200

    Default port policy to force replacement

    This change adds a replacement_policy property to OS::Neutron::Port
    which defaults to always replacing the port on stack-update
    regardless of any changes to the template.

    Currently when a server is replaced, the new server is booted with
    the port that is still attached to the old server, which raises a
    port-still-in-use error.

    Even if heat managed to detach and attach a single port, Nova currently
    deletes all ports on interface-detach and server delete, so the port
    would no longer be available anyway.

    This change ensures that a new server is always booted with a new port,
    so it fixes the above 2 issues, however there are implications for
    other scenarios.

    If the server is not replaced during the stack-update, the server
    handle_update will detach/attach to the new port, so this is fine.

    If the port specifies a fixed_ips ip_address which doesn't change
    during stack-update, an error will be raised that 2 ports exist
    with the same IP address. The only current workaround would be
    to set update_policy:AUTO and not make any changes to the server
    which results in server replacement (or do 2 stack updates using
    a transition ip_address).

    Likewise, update_policy:AUTO will need to be set on any port
    consumed by resources which don't support update without replacement
    (such as OS::Neutron::FloatingIP and OS::Trove::Instance)

    Change-Id: I558db6bac196f49e5c488a577f0580c934b06747
    Closes-Bug: #1301486
    Related-Bug: #1158684
    Related-Bug: #1369748

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/126309

[OpenStack Infra (hudson-openstack)]

Change abandoned by Aaron Rosen (<email address hidden>) on branch: master
Review: https://review.openstack.org/77043
Reason: replaced by https://review.openstack.org/#/c/126309/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/126740
Committed: https://git.openstack.org/cgit/stackforge/group-based-policy/commit/?id=4cb47e63ac02673914c1ce68dd2482930929575b
Submitter: Jenkins
Branch: master

commit 4cb47e63ac02673914c1ce68dd2482930929575b
Author: Ivar Lazzaro <email address hidden>
Date: Tue Oct 7 14:59:35 2014 -0700

    Delete object chain

    Reversing the order in which drivers are called for object
    deletion.

    Change-Id: Ia9592fcadd7480ea1c6ebe048e712c587d0bdac3
    Closes-bug: #1378530
    Workarounds-bug: #1158684

[Robert Kukura (rkukura)]

Ivar's patch in comment 15 works around this Nova bug by changing the schema to set the port_id foreign key to null on delete. I think this is sufficient for the Juno GBP release.

Alternatively, it should be possible to add an ML2 mechanism driver that "intercepts" port delete and deletes the corresponding GBP policy target. Ivar's work-around is preferable for at least two reasons: 1) It is independent of the core plugin whereas this would only work with ML2, and 2) Nova should not be deleting the port (and/or policy target) when it was created externally to Nova.

Once the Nova bug is fixed, we should change schema so the port_id foreign key is no longer be set to null on delete.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/126309
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Robert Kukura (rkukura)]

I don't think there is anything we need to do on the GBP side until https://review.openstack.org/#/c/126309/ is merged in nova, but that seems to be on track for kilo-2 (or very soon thereafter).

[Davanum Srinivas (DIMS) (dims-v)]

Fix proposed again

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/126309
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1153a46738fc3ffff98a1df9d94b5a55fdd58777
Submitter: Jenkins
Branch: master

commit 1153a46738fc3ffff98a1df9d94b5a55fdd58777
Author: Boden R <email address hidden>
Date: Fri Oct 3 13:48:08 2014 -0400

    Preserve preexisting ports on server delete

    When using the '--nic port-id' option for nova boot to bind
    a server to an existing port, nova deletes the preexisting
    port on server terminate (delete). This behavior is not correct
    since the user has preallocated the port object and will
    expect it to only be unbound from the server on delete.

    This patch allow the existing port to be unbound from the instance
    on server delete whereupon the port will still exist but not
    be associated with the instance in neutron (note the '--nic
    port-id' opt is only supported with neutron). The patch also
    adds unit tests to verify the behavior + changes.

    This patch also moves instance.info_cache.delete() to be called
    after _shutdown in the compute manager as shutdown calls
    deallocate_for_instance so the info_cache is still needed at this point.

    Based on original work from Aaron Rosen <email address hidden>
    for which you can see a stale patch review here:
    https://review.openstack.org/#/c/77043/

    UpgradeImpact:

    Existing applications that created neutron ports and assigned them to a running
    instance or to a instance that is created will need to be aware
    that they are now respponsible for the deletion of the ports and that these
    ports will not be deleted by nova when the instance is destroyed.

    Co-authored-by: Aaron Rosen <email address hidden>
    Co-authored-by: Davanum Srinivas <email address hidden>

    Change-Id: Ia5367cf064d40690670ffeac3c1f16998464c234
    Closes-bug: 1158684

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/169543

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/169543
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=2b4a4010821ebb242806ae6ff4fc1ad4db62fecb
Submitter: Jenkins
Branch: master

commit 2b4a4010821ebb242806ae6ff4fc1ad4db62fecb
Author: Steve Baker <email address hidden>
Date: Wed Apr 1 11:33:49 2015 +1300

    OS::Neutron::Port default replacement_policy=AUTO

    Now that nova bug 1158684 has been fixed heat can assume that a port
    won't be deleted by nova during a stack update.

    REPLACE_ALWAYS as a default caused its share of issues, including
    servers not handling the port churn on stack updates.

    REPLACE_ALWAYS is left as an option for standalone heat orchestrating
    older OpenStack releases.

    Change-Id: Ie9b2ebe8b29bd5ed6006dfb59c6dccd4a595832f
    Closes-Bug: #1393376
    Related-Bug: #1158684

[Ethan Lynn (ethanlynn)]

Is there any plan to backport this patch to juno?

[rick jones (perfgeek)]

Horses, barn doors and pedantry perhaps, but I trust this change in behaviour has been copiously documented somewhere moderately prominent? I'm content with the change, but the behaviour has been "the other way" (ports deleted) for a while and for example, I had become habituated to it. Perhaps even counting on it (though not that strongly). I rather doubt I am unique in that regard.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/215160

[Robert Kukura (rkukura)]

The nova backport to stable/juno linked in comment 25 is currently WIP, and I'm not sure if/when it will merge. In the mean time, distributions supporting GBP may wish to include this patch in nova.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Roman Podoliaka (<email address hidden>) on branch: stable/juno
Review: https://review.openstack.org/215160

[Robert Kukura (rkukura)]

This is fixed in nova from kilo onward, but it doesn't look like the fix will be back-ported to juno. I think we should just close the GBP bug at this point and live with the workarounds we have in juno.

[Robert Kukura (rkukura)]

Closing as won't fix in GBP.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/327046
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=9088891ce72081684761e5cf54d3b3eabab0ca37
Submitter: Jenkins
Branch: master

commit 9088891ce72081684761e5cf54d3b3eabab0ca37
Author: Sam Betts <email address hidden>
Date: Wed Nov 30 18:29:04 2016 +0000

    Add Virtual Network Interface Driver APIs

    This patch adds the driver API interfaces for the virtual network
    interface API in order to abstract the task of assigning logical network
    interfaces to physical network interfaces.

    Since the OpenStack Newton release, Ironic provides an interface for
    pluggable network implementations. Different network implementations may
    want to handle how logical to physical network interface assignment
    happens. To do this the new API calls into new functions on the network
    implementation loaded for the specified node.

    This is part 1 of 3, and adds four new functions vif_attach, vif_detach,
    vif_list, port_changed, portgroup_changed, get_current_vif to the base
    network interface class, which should be overridden by network interface
    implementations.

    DHCP provider update_mac_address method was deprecated, network
    interface port_changed() and portgroup_changed() should be used instead.

    Co-Authored-By: Vasyl Saienko (<email address hidden>)
    Change-Id: I0b84cfd85557d18254697f2e539c583ea0f8e88c
    Partial-Bug: #1582188
    Closes-Bug: #1158684

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ironic 7.0.0 release.