Bandwidth resource is leaked if a bound port is deleted in neutron

[Balazs Gibizer]

1) Boot a server with a neutron port having QoS min bandwidth rule
2) Delete the bound port in neutron
3) Nova detaches the port form the Server but does not update the allocation in Placement

So the bandwidth resource has been leaked. However the leaked resource can be recovered by deleting the Server.

[Balazs Gibizer (balazs-gibizer)]

Another possible workaround is to detach the port via nova API and then delete the port in neutron.

[Balazs Gibizer (balazs-gibizer)]

@Melanie: I don't think this is an RC blocker problem:
1) There is a clear workaround, detach the port via the nova API then delete the neutron port.
2) I only heard about one use case when the above direct delete might be used instead of the two step detach + delete. A nova server stuck in a wrong state where port detach cannot be used to free up the SRIOV VF and the admin needs that VF so she goes and deletes the port in neutron using that VF. However I don't think that in this case we need to expect that everything is cleaned up properly. When such stuck Server is finally deleted nova will clean up the resource allocation.

[Balazs Gibizer (balazs-gibizer)]

Btw, added a PTG cross project topic regarding this bug to https://etherpad.openstack.org/p/nova-ptg-train

[melanie witt (melwitt)]

Thanks gibi. I'll remove the stein-rc-potential tag accordingly. I wonder if we should add the information about the workaround to the release notes Known Issues.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/644694

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/644694
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=70989c3eb508ac4633a921a3760e02a4b74a9784
Submitter: Zuul
Branch: master

commit 70989c3eb508ac4633a921a3760e02a4b74a9784
Author: melanie witt <email address hidden>
Date: Tue Mar 19 22:50:48 2019 +0000

    Add known issue for minimum bandwidth resource leak

    Nova will leak minimum bandwidth resources in placement if a user
    deletes a bound port from Neutron out-of-band. This adds a note about
    how users can work around the issue.

    Related-Bug: #1820588

    Change-Id: I41f42c1a7595d9e6a73d1261bf1ac1d47ddadcdf

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/657079

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/657079
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e9d572ddfad10da18603c55da1aad2e70da15381
Submitter: Zuul
Branch: master

commit e9d572ddfad10da18603c55da1aad2e70da15381
Author: Balazs Gibizer <email address hidden>
Date: Fri May 3 16:14:48 2019 -0600

    Log when port resource is leaked during port delete

    When a bound port is deleted in neutron, nova gets a
    network-vif-deleted notification and the port is
    detached from the server. However if the port had resource request
    then the resulting allocation is leaked.

    This patch makes sure nova at least logs an ERROR. Also this patch
    asserts that the leaked allocation is reclaimed when the server is
    deleted.

    Change-Id: I5d905aeb5b25f84d406dbf238d0d3a46f0f81161
    Related-Bug: #1820588

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/657581

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/657581
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a2b1ef5f11f38a3359613babff86ec861bff4373
Submitter: Zuul
Branch: stable/stein

commit a2b1ef5f11f38a3359613babff86ec861bff4373
Author: Balazs Gibizer <email address hidden>
Date: Fri May 3 16:14:48 2019 -0600

    Log when port resource is leaked during port delete

    When a bound port is deleted in neutron, nova gets a
    network-vif-deleted notification and the port is
    detached from the server. However if the port had resource request
    then the resulting allocation is leaked.

    This patch makes sure nova at least logs an ERROR. Also this patch
    asserts that the leaked allocation is reclaimed when the server is
    deleted.

    Change-Id: I5d905aeb5b25f84d406dbf238d0d3a46f0f81161
    Related-Bug: #1820588
    (cherry picked from commit e9d572ddfad10da18603c55da1aad2e70da15381)