add-router-interface can not recover when failing create_port_post_commit

Bug #1525163 reported by Yushiro FURUKAWA
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Undecided
Yushiro FURUKAWA

Bug Description

When the following situation, a port created that no one can delete it:

  1. Configure ML2 plugin.
  2. Execute router-interface-add with subnet_id and failed in create_port_post_commit

  After that, undeletable port is created.
  Even if the admin tries to delete the port, he cannot delete it.

    case1: delete-port with port_id
      => Port {port_id} cannot be deleted directly via the port API: has device owner network:router_interface.

    case2: router-interface-delete with subnet_id or port_id
      => Router {router_id} has no interface on subnet {subnet_id}
      => Router {router_id} does not have an interface with id {port_id}

[How to fix]
  create_port_post_commit has a recovery process. When failed, then calls
  delete_port. However, in this case, 'device_owner' and 'device_id' have already
  registered at port's DB. Therefore, delete_port fails due to device_owner check.

  Hence, I'll add 'l3_port_check=False' into delete_port's argument.

[Environment]
  trunk(devstack all-in-one with ML2 plugin(openvswitch))

[How to reproduce]
* You have to arrange create_port_post_commit shuld be failed.

source devstack/openrc admin admin
export TOKEN=`openstack token issue | grep ' id ' | get_field 2`

curl -s -X GET -H "x-auth-token:$TOKEN" 192.168.122.253:9696/v2.0/ports | jq "."
{
  "ports": []
}

curl -i -X PUT -d '{"subnet_id":"214ebeb5-2d08-4ae5-9d60-3c7a76d56746"}' -H "x-auth-token:$TOKEN" 192.168.122.253:9696/v2.0/routers/7d1561d1-71f9-4355-9248-5ac313de8ee3/add_router_interface

HTTP/1.1 409 Conflict
Content-Type: application/json; charset=UTF-8
Content-Length: 204
X-Openstack-Request-Id: req-ec3bad1f-84a2-4865-9cac-e63723c0a3bb
Date: Fri, 11 Dec 2015 10:11:11 GMT

{"NeutronError": {"message": "Port 570a4166-d463-4ee6-894b-f8aab6cc63b2 cannot be deleted directly via the port API: has device owner network:router_interface.", "type": "ServicePortInUse", "detail": ""}}

$ curl -s -X GET -H "x-auth-token:$TOKEN" 192.168.122.253:9696/v2.0/ports/570a4166-d463-4ee6-894b-f8aab6cc63b2 | jq "."
{
  "port": {
    "mac_address": "fa:16:3e:c3:1c:8d",
    "tenant_id": "4c0b8881d3e24a1cb1afe9ea6b07d946",
    "binding:vif_type": "unbound",
    "binding:vnic_type": "normal",
    "binding:vif_details": {},
    "binding:profile": {},
    "port_security_enabled": false,
    "device_owner": "network:router_interface",
    "dns_assignment": [
      {
        "fqdn": "host-172-16-1-1.openstacklocal.",
        "ip_address": "172.16.1.1",
        "hostname": "host-172-16-1-1"
      }
    ],
    "extra_dhcp_opts": [],
    "allowed_address_pairs": [],
    "binding:host_id": "",
    "status": "DOWN",
    "fixed_ips": [
      {
        "ip_address": "172.16.1.1",
        "subnet_id": "214ebeb5-2d08-4ae5-9d60-3c7a76d56746"
      }
    ],
    "id": "570a4166-d463-4ee6-894b-f8aab6cc63b2",
    "security_groups": [],
    "device_id": "7d1561d1-71f9-4355-9248-5ac313de8ee3",
    "name": "",
    "admin_state_up": true,
    "network_id": "11515598-c20e-4e8a-94d0-1fef56f4607d",
    "dns_name": ""
  }
}

Tags: neutron
Changed in neutron:
assignee: nobody → Yushiro FURUKAWA (y-furukawa-2)
Revision history for this message
Isaku Yamahata (yamahata) wrote :

Here is the stack trace when postcommit failed.

2016-01-06 11:26:59.812 ERROR oslo_db.api [req-285516b2-87b3-46a8-9898-162d0796a52b tempest-RoutersTest-602986699 46cbce09c21b4d8195477acd8fe8e71b] DB error.
2016-01-06 11:26:59.812 TRACE oslo_db.api Traceback (most recent call last):
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 137, in wrapper
2016-01-06 11:26:59.812 TRACE oslo_db.api return f(*args, **kwargs)
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/api/v2/base.py", line 217, in _handle_action
2016-01-06 11:26:59.812 TRACE oslo_db.api ret_value = getattr(self._plugin, name)(*arg_list, **kwargs)
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/networking-odl/networking_odl/l3/l3_odl.py", line 135, in add_router_interface
2016-01-06 11:26:59.812 TRACE oslo_db.api context, router_id, interface_info)
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/db/l3_dvr_db.py", line 264, in add_router_interface
2016-01-06 11:26:59.812 TRACE oslo_db.api context, router, interface_info['subnet_id'], device_owner)
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/db/l3_db.py", line 628, in _add_interface_by_subnet
2016-01-06 11:26:59.812 TRACE oslo_db.api {'port': port_data}), [subnet], True
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/plugins/common/utils.py", line 149, in create_port
2016-01-06 11:26:59.812 TRACE oslo_db.api return core_plugin.create_port(context, {'port': port_data})
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/plugins/ml2/plugin.py", line 1061, in create_port
2016-01-06 11:26:59.812 TRACE oslo_db.api self.delete_port(context, result['id'])
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/plugins/ml2/plugin.py", line 1334, in delete_port
2016-01-06 11:26:59.812 TRACE oslo_db.api self._pre_delete_port(context, id, l3_port_check)
2016-01-06 11:26:59.812 TRACE oslo_db.api File "/home/yamahata/openstack/odl/neutron/neutron/plugins/ml2/plugin.py", line 1330, in _pre_delete_port
2016-01-06 11:26:59.812 TRACE oslo_db.api raise e.errors[0].error
2016-01-06 11:26:59.812 TRACE oslo_db.api ServicePortInUse: Port c1de21c0-796b-456b-a19a-b6b58425f43b cannot be deleted directly via the port API: has device owner network:router_interface.
2016-01-06 11:26:59.812 TRACE oslo_db.api

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/264662

Changed in neutron:
assignee: Yushiro FURUKAWA (y-furukawa-2) → Isaku Yamahata (yamahata)
status: New → In Progress
Changed in neutron:
assignee: Isaku Yamahata (yamahata) → Yushiro FURUKAWA (y-furukawa-2)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/264662
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=20fbd1d0cda8cef62c9f1fc913ce7bd4688a381f
Submitter: Jenkins
Branch: master

commit 20fbd1d0cda8cef62c9f1fc913ce7bd4688a381f
Author: Isaku Yamahata <email address hidden>
Date: Thu Jan 7 00:12:16 2016 -0800

    Modify an option for delete-port as recovery

    When create_port failed on add_router_interface, it fails to delete
    the port. On error recovery path, l3_port_check=False should be passed
    to delete_port().

    Partial-bug: #1525163
    Co-Authored-By: Yushiro FURUKAWA <email address hidden>
    Change-Id: I5f83c99aaa2ebc061630a8382fe98e4460e7f620

Revision history for this message
Yushiro FURUKAWA (y-furukawa-2) wrote :

Hi,

This bug-report has been committed. Would you change the status from "In progress" to "Fix xxxx", please?

Changed in neutron:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.