PortBindingFailed_Remote (HTTP 500)

[Christos Tsalidis]

Hi all,

Description
===========

I am running newton on centos7 and have currently one active instance. This instance is connected to two vxlan networks(int and vxlannetwork) and I would like to get it connected to two additonal, vlan and gre. Networks are created, but after I create the port and try to attach the interface I get an error. See below.

Steps to reproduce
===================

# nova list
+--------------------------------------+--------+--------+------------+-------------+---------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+--------+--------+------------+-------------+---------------------------------------+
| 354caa3d-bddb-4396-ac0a-ef12e06b14e3 | cirros | ACTIVE | - | Running | int=10.0.0.5; vxlannetwork=10.0.10.12 |
+--------------------------------------+--------+--------+------------+-------------+---------------------------------------+

when I create a port on vlan network and try to attach I get the following error

# neutron net-list
+--------------------------------------+--------------+----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+--------------+----------------------------------------------------+
| 625914e4-c490-4808-a309-8591c97d62ea | ext | 6dc90b28-4b84-456f-bc95-1b3161c77ab5 10.0.100.0/24 |
| 7974307b-d4e8-4b6b-9354-62ffb0d148e2 | vxlannetwork | e0494841-73dd-4c35-ad07-b8b7a0ae6fdb 10.0.10.0/24 |
| afd58b60-64c4-4ed3-9da9-c107bf40f593 | vlannetwork | 2549725d-9847-4d03-a9ab-02be5da3907b 10.0.20.0/24 |
| da177468-f3d8-45ef-a9d7-4ad03960dfab | int | 934d2862-bdb0-4369-a290-3e8dae99997c 10.0.0.0/24 |
| de00479b-0777-4589-b20c-efe0858a9041 | grenetwork | e9ad3685-3935-4c5c-bc6a-64d8fe0f09c3 10.0.30.0/24 |
+--------------------------------------+--------------+----------------------------------------------------+

#neutron port-create afd58b60-64c4-4ed3-9da9-c107bf40f593
Created a new port:
+-----------------------+----------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+----------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | unbound |
| binding:vnic_type | normal |
| created_at | 2017-11-06T19:00:40Z |
| description | |
| device_id | |
| device_owner | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "2549725d-9847-4d03-a9ab-02be5da3907b", "ip_address": "10.0.20.9"} |
| id | 2795d640-adfe-417e-a598-68b7336b19fa |
| mac_address | fa:16:3e:cc:27:f9 |
| name | |
| network_id | afd58b60-64c4-4ed3-9da9-c107bf40f593 |
| project_id | 2dda4fa3451947808fec2b15ace75719 |
| revision_number | 4 |
| security_groups | 68c3297c-82b9-4a03-9f7c-6e9d904d143a |
| status | DOWN |
| tenant_id | 2dda4fa3451947808fec2b15ace75719 |
| updated_at | 2017-11-06T19:00:40Z |
+-----------------------+----------------------------------------------------------------------------------+
# nova interface-attach --port-id 2795d640-adfe-417e-a598-68b7336b19fa cirros
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'PortBindingFailed_Remote'> (HTTP 500) (Request-ID: req-fb7d1509-c001-498b-ab01-6aeb41cfc1ae)

# cat /var/log/nova/nova-api.log | grep ERROR

2017-11-06 20:01:05.901 8536 ERROR nova.api.openstack.extensions PortBindingFailed: Binding failed for port 2795d640-adfe-417e-a598-68b7336b19fa, please check neutron logs for more information.

# cat /var/log/neutron/server.log | grep ERROR

2017-11-06 20:01:05.821 19663 ERROR neutron.plugins.ml2.managers [req-73218b7a-0136-47e5-bf4d-1f1ddc3a121e dc8bad16bb5a4ab480008f630815fb1c ccfc1d1da08b42af976c75cac75b62d5 - - -] Failed to bind port 2795d640-adfe-417e-a598-68b7336b19fa on host cl310openstack.lab.local for vnic_type normal using segments [{'segmentation_id': 35, 'physical_network': u'physnet1', 'id': u'6b180747-3063-480b-b752-af94a67ca7f5', 'network_type': u'vlan'}]

# cat /etc/neutron/plugins/ml2/ml2_conf.ini
[DEFAULT]
[ml2]
type_drivers = vxlan,vlan,gre
tenant_network_types = vxlan,vlan,gre
mechanism_drivers =openvswitch
path_mtu = 0
[ml2_type_flat]
[ml2_type_geneve]
[ml2_type_gre]
tunnel_id_ranges =10:100
[ml2_type_vlan]
network_vlan_ranges =physnet1:1:100
[ml2_type_vxlan]
vni_ranges =10:100
vxlan_group = 224.0.0.1
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True

Expected result
===============

Expected results would be to attach the additional networks, vlan and gre to instance.

Environment
===========

centos7
libvirt version: 3.2.0, package: 14.el7_4.3 (CentOS BuildSystem <http://bugs.centos.org>, 2017-09-07-11:27:44, c1bm.rdu2.centos.org)
Ceph Storage
Neutron with OpenVSwitch

Many thanks!

[Armando Migliaccio (armando-migliaccio)]

The error:

Failed to bind port 2795d640-adfe-417e-a598-68b7336b19fa on host cl310openstack.lab.local for vnic_type normal using segments [{'segmentation_id': 35, 'physical_network': u'physnet1', 'id': u'6b180747-3063-480b-b752-af94a67ca7f5', 'network_type': u'vlan'}

May be the result of an incompatible configuration. The ID 6b180747-3063-480b-b752-af94a67ca7f5 is it accessible to the end-user?

[Matt Riedemann (mriedem)]

The compute API should at least handle the PortBindingFailed error and not produce a 500 to the user, it's more likely a 400 error.

[Matt Riedemann (mriedem)]

On the other hand, nova really has no idea why the port binding failed, we don't know if it's a bad request or an actual failure on the networking backend.

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for OpenStack Compute (nova) because there has been no activity for 60 days.]