After some digging around, and reading LP:1822884, which may be related or the same bug, I found that there were extra entries in the ml2_port_bindings and ml2_port_binding_levels tables for the port in question:
After removing the INACTIVE rows from the first table, and the entries for every host except the host the instances was running on from the second, I was able to successfully invoke a migration, which failed due to lack of resources on the destination host:
$ openstack server migrate --block-migration 8703d9db-81b0-4e86-a2ef-c4ba5250556c --live shinx --disk-overcommit --wait
No valid host was found. Unable to move instance 8703d9db-81b0-4e86-a2ef-c4ba5250556c to host shinx. There is not enough capacity on the host for the instance. (HTTP 400) (Request-ID: req-70e5caea-8fa9-4314-b2b2-a7ae190179a6)
Attempting the migration again:
jujumanage@corsola:~$ openstack server migrate --block-migration 8703d9db-81b0-4e86-a2ef-c4ba5250556c --live shinx --disk-overcommit --wait
Migration pre-check error: Binding failed for port 5a3c5d23-8727-47d2-af72-a53b495358d2, please check neutron logs for more information. (HTTP 400) (Request-ID: req-eb051a44-a965-4df
4-9b3b-14ac3a30acf8)
And indeed the database is back in a similar state as before:
mysql> select * from ml2_port_bindings where port_id = '5a3c5d23-8727-47d2-af72-a53b495358d2';
+--------------------------------------+---------+----------+-----------+---------+---------------------------------------------------------------------------------------------------
-+----------+
| port_id | host | vif_type | vnic_type | profile | vif_details
| status |
+--------------------------------------+---------+----------+-----------+---------+---------------------------------------------------------------------------------------------------
-+----------+
| 5a3c5d23-8727-47d2-af72-a53b495358d2 | shinx | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | INACTIVE |
| 5a3c5d23-8727-47d2-af72-a53b495358d2 | sliggoo | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"}
| ACTIVE |
+--------------------------------------+---------+----------+-----------+---------+---------------------------------------------------------------------------------------------------
-+----------+
2 rows in set (0.00 sec)
After some digging around, and reading LP:1822884, which may be related or the same bug, I found that there were extra entries in the ml2_port_bindings and ml2_port_ binding_ levels tables for the port in question:
mysql> select * from ml2_port_bindings where port_id = '5a3c5d23- 8727-47d2- af72-a53b495358 d2'; ------- ------- ------- ------- ----+-- ------- +------ ----+-- ------- --+---- -----+- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -+----- -----+ ------- ------- ------- ------- ----+-- ------- +------ ----+-- ------- --+---- -----+- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -+----- -----+ 8727-47d2- af72-a53b495358 d2 | happiny | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | INACTIVE | 8727-47d2- af72-a53b495358 d2 | jynx | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | INACTIVE | 8727-47d2- af72-a53b495358 d2 | shinx | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | INACTIVE | 8727-47d2- af72-a53b495358 d2 | sliggoo | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | ACTIVE | ------- ------- ------- ------- ----+-- ------- +------ ----+-- ------- --+---- -----+- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -+----- -----+
+------
| port_id | host | vif_type | vnic_type | profile | vif_details | status |
+------
| 5a3c5d23-
| 5a3c5d23-
| 5a3c5d23-
| 5a3c5d23-
+------
4 rows in set (0.00 sec)
mysql> select * from ml2_port_ binding_ levels where port_id = '5a3c5d23- 8727-47d2- af72-a53b495358 d2';
+------ ------- ------- ------- ------- ----+-- ------- +------ -+----- ------- -+----- ------- ------- ------- ------- -----+ ------- ------- ------- ------- ----+-- ------- +------ -+----- ------- -+----- ------- ------- ------- ------- -----+ 8727-47d2- af72-a53b495358 d2 | happiny | 0 | openvswitch | afaa634a- dff5-437e- a7a0-8630c21b2a 73 | 8727-47d2- af72-a53b495358 d2 | jynx | 0 | openvswitch | afaa634a- dff5-437e- a7a0-8630c21b2a 73 | 8727-47d2- af72-a53b495358 d2 | shinx | 0 | openvswitch | afaa634a- dff5-437e- a7a0-8630c21b2a 73 | 8727-47d2- af72-a53b495358 d2 | sliggoo | 0 | openvswitch | afaa634a- dff5-437e- a7a0-8630c21b2a 73 | ------- ------- ------- ------- ----+-- ------- +------ -+----- ------- -+----- ------- ------- ------- ------- -----+
| port_id | host | level | driver | segment_id |
+------
| 5a3c5d23-
| 5a3c5d23-
| 5a3c5d23-
| 5a3c5d23-
+------
4 rows in set (0.00 sec)
After removing the INACTIVE rows from the first table, and the entries for every host except the host the instances was running on from the second, I was able to successfully invoke a migration, which failed due to lack of resources on the destination host:
$ openstack server migrate --block-migration 8703d9db- 81b0-4e86- a2ef-c4ba525055 6c --live shinx --disk-overcommit --wait 81b0-4e86- a2ef-c4ba525055 6c to host shinx. There is not enough capacity on the host for the instance. (HTTP 400) (Request-ID: req-70e5caea- 8fa9-4314- b2b2-a7ae190179 a6)
No valid host was found. Unable to move instance 8703d9db-
Attempting the migration again:
jujumanage@ corsola: ~$ openstack server migrate --block-migration 8703d9db- 81b0-4e86- a2ef-c4ba525055 6c --live shinx --disk-overcommit --wait 8727-47d2- af72-a53b495358 d2, please check neutron logs for more information. (HTTP 400) (Request-ID: req-eb051a44- a965-4df 14ac3a30acf8)
Migration pre-check error: Binding failed for port 5a3c5d23-
4-9b3b-
And indeed the database is back in a similar state as before:
mysql> select * from ml2_port_bindings where port_id = '5a3c5d23- 8727-47d2- af72-a53b495358 d2'; ------- ------- ------- ------- ----+-- ------- +------ ----+-- ------- --+---- -----+- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ----+-- ------- +------ ----+-- ------- --+---- -----+- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- 8727-47d2- af72-a53b495358 d2 | shinx | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | INACTIVE | 8727-47d2- af72-a53b495358 d2 | sliggoo | ovs | normal | | {"port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} ------- ------- ------- ------- ----+-- ------- +------ ----+-- ------- --+---- -----+- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -------
+------
-+----------+
| port_id | host | vif_type | vnic_type | profile | vif_details
| status |
+------
-+----------+
| 5a3c5d23-
| 5a3c5d23-
| ACTIVE |
+------
-+----------+
2 rows in set (0.00 sec)
mysql> select * from ml2_port_ binding_ levels where port_id = '5a3c5d23- 8727-47d2- af72-a53b495358 d2'; ------- ------- ------- ------- ----+-- ------- +------ -+----- ------- -+----- ------- ------- ------- ------- -----+ ------- ------- ------- ------- ----+-- ------- +------ -+----- ------- -+----- ------- ------- ------- ------- -----+ 8727-47d2- af72-a53b495358 d2 | shinx | 0 | openvswitch | afaa634a- dff5-437e- a7a0-8630c21b2a 73 | 8727-47d2- af72-a53b495358 d2 | sliggoo | 0 | openvswitch | afaa634a- dff5-437e- a7a0-8630c21b2a 73 | ------- ------- ------- ------- ----+-- ------- +------ -+----- ------- -+----- ------- ------- ------- ------- -----+
+------
| port_id | host | level | driver | segment_id |
+------
| 5a3c5d23-
| 5a3c5d23-
+------
2 rows in set (0.00 sec)
mysql>