| 2021-11-19 16:48:42 |
Alexander Shishebarov |
bug |
|
|
added bug |
| 2021-11-20 12:23:56 |
Alexander Shishebarov |
description |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server migration with a disabled port(admin_state_up DWON)/
Here ports configuration
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| 09d5db66-1f89-41dd-a215-74888f3099f3 | | fa:16:3e:98:30:50 | ip_address='10.1.1.1', subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
| df08c6b0-ca45-4985-8e5f-8fb95f904ac6 | | fa:16:3e:5c:eb:f4 | ip_address='192.168.0.7', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what vm
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for an event on each port, regardless of its initial state.
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send a message if the port is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server migration with a disabled port(admin_state_up DWON)/
Here ports configuration
```
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| 09d5db66-1f89-41dd-a215-74888f3099f3 | | fa:16:3e:98:30:50 | ip_address='10.1.1.1', subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
| df08c6b0-ca45-4985-8e5f-8fb95f904ac6 | | fa:16:3e:5c:eb:f4 | ip_address='192.168.0.7', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
```
When we try migrate what vm
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for an event on each port, regardless of its initial state.
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send a message if the port is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py |
|
| 2021-11-20 12:41:03 |
Alexander Shishebarov |
description |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server migration with a disabled port(admin_state_up DWON)/
Here ports configuration
```
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| 09d5db66-1f89-41dd-a215-74888f3099f3 | | fa:16:3e:98:30:50 | ip_address='10.1.1.1', subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
| df08c6b0-ca45-4985-8e5f-8fb95f904ac6 | | fa:16:3e:5c:eb:f4 | ip_address='192.168.0.7', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
```
When we try migrate what vm
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for an event on each port, regardless of its initial state.
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send a message if the port is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server live migration with at least one administratively disabled port(admin_state_up DWON).
live_migration_wait_for_vif_plug is enabled by default.
Here ports configuration of VM.
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| e09dca39-f62f-4a3b-a0f6-4d98edcd037e | | fa:16:3e:83:73:13 | ip_address='192.168.0.3', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
| e331b3d3-cf59-49a0-b531-590433523f6f | | fa:16:3e:53:b4:3c | ip_address=’10.10.10.1, subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what VM
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for the network-vif-plugged event for each port of migrating VM, regardless of its initial state(up or down).
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send the rpc message with "network-vif-plugged" if the port of migrating VM is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py
As a result, the migration process is stopped. |
|
| 2021-11-20 12:42:44 |
Alexander Shishebarov |
description |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server live migration with at least one administratively disabled port(admin_state_up DWON).
live_migration_wait_for_vif_plug is enabled by default.
Here ports configuration of VM.
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| e09dca39-f62f-4a3b-a0f6-4d98edcd037e | | fa:16:3e:83:73:13 | ip_address='192.168.0.3', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
| e331b3d3-cf59-49a0-b531-590433523f6f | | fa:16:3e:53:b4:3c | ip_address=’10.10.10.1, subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what VM
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for the network-vif-plugged event for each port of migrating VM, regardless of its initial state(up or down).
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send the rpc message with "network-vif-plugged" if the port of migrating VM is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py
As a result, the migration process is stopped. |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server live migration with at least one administratively disabled port(admin_state_up DWON).
live_migration_wait_for_vif_plug is enabled by default.
Here ports configuration of VM.
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| e09dca39-f62f-4a3b-a0f6-4d98edcd037e | | fa:16:3e:83:73:13 | ip_address='192.168.0.3', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
| e331b3d3-cf59-49a0-b531-590433523f6f | | fa:16:3e:53:b4:3c | ip_address=’10.10.10.1, subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what VM
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed.
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for the network-vif-plugged event for each port of migrating VM, regardless of its initial state(up or down).
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send the rpc message with "network-vif-plugged" if the port of migrating VM is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py
As a result, the migration process is stopped. |
|
| 2021-11-20 12:43:25 |
Alexander Shishebarov |
description |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server live migration with at least one administratively disabled port(admin_state_up DWON).
live_migration_wait_for_vif_plug is enabled by default.
Here ports configuration of VM.
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| e09dca39-f62f-4a3b-a0f6-4d98edcd037e | | fa:16:3e:83:73:13 | ip_address='192.168.0.3', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
| e331b3d3-cf59-49a0-b531-590433523f6f | | fa:16:3e:53:b4:3c | ip_address=’10.10.10.1, subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what VM
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed.
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
This happens because nova-compute is waiting for the network-vif-plugged event for each port of migrating VM, regardless of its initial state(up or down).
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send the rpc message with "network-vif-plugged" if the port of migrating VM is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py
As a result, the migration process is stopped. |
We use neutron(stable/stein) ml2/ovs plugin and nova (stable/stein)
An error occurs in case of server live migration with at least one administratively disabled port(admin_state_up DWON).
live_migration_wait_for_vif_plug is enabled by default.
Here ports configuration of VM.
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| ID | Name | MAC Address | Fixed IP Addresses | Status |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
| e09dca39-f62f-4a3b-a0f6-4d98edcd037e | | fa:16:3e:83:73:13 | ip_address='192.168.0.3', subnet_id='0bb936ed-c4a4-4a5d-be18-2794a73aea79' | DOWN |
| e331b3d3-cf59-49a0-b531-590433523f6f | | fa:16:3e:53:b4:3c | ip_address=’10.10.10.1, subnet_id='a6786536-b67b-40a4-9470-e3b158a71dbc' | ACTIVE |
+--------------------------------------+------+-------------------+-----------------------------------------------------------------------------+--------+
When we try migrate what VM
openstack server migrate b4743fab-17e0-48af-8ad3-3b81fd05a968 --live cmp1
An error occurs in the pre live migration process.
The error occurs on the server from which the migration is performed.
"
2021-11-18 17:34:28,910.910 2173136 WARNING nova.compute.manager [-] [instance: b4743fab-17e0-48af-8ad3-3b81fd05a968] Timed out waiting for events: [('network-vif-plugged', u'e09dca39-f62f-4a3b-a0f6-4d98edcd037e'), ('network-vif-plugged', u'e331b3d3-cf59-49a0-b531-590433523f6f')]. If these timeouts are a persistent issue it could mean the networking backend on host cmp1 does not support sending these events unless there are port binding host changes which does not happen at this point in the live migration process. You may need to disable the live_migration_wait_for_vif_plug option on host cmp1.: Timeout: 300 seconds
"
This happens because nova-compute is waiting for the network-vif-plugged event for each port of migrating VM, regardless of its initial state(up or down).
https://github.com/openstack/nova/blob/stable/stein/nova/compute/manager.py#L6767
But the neutron server does not send the rpc message with "network-vif-plugged" if the port of migrating VM is disabled.
https://github.com/openstack/neutron/blob/stable/stein/neutron/notifiers/nova.py#L207
"
2021-11-18 16:09:03,642.642 2916237 DEBUG neutron.notifiers.nova [req-98b397eb-db00-4370-9510-b071c501a12e b6ba9c75146a49829a7427a3e8cc3c10 192796e61c174f718d6147b129f3f2ff - default default] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id e09dca39-f62f-4a3b-a0f6-4d98edcd037e record_port_status_changed /usr/lib/python2.7/dist-packages/neutron/notifiers/nova.py
"
As a result, the migration process is stopped. |
|
| 2021-11-23 15:10:40 |
Sylvain Bauza |
nova: status |
New |
Confirmed |
|
| 2021-11-23 15:10:45 |
Sylvain Bauza |
nova: importance |
Undecided |
Low |
|
| 2021-11-23 15:11:00 |
Sylvain Bauza |
tags |
|
neutron |
|
| 2021-11-26 20:59:39 |
OpenStack Infra |
nova: status |
Confirmed |
In Progress |
|
| 2022-02-21 07:28:21 |
Danila Balagansky |
bug |
|
|
added subscriber Danila Balagansky |
| 2022-12-05 12:12:20 |
s10 |
bug |
|
|
added subscriber s10 |
