| 2015-02-11 01:28:01 |
Liang Chen |
bug |
|
|
added bug |
| 2015-02-12 00:08:44 |
Chris Johnston |
neutron: status |
New |
Confirmed |
|
| 2015-02-12 00:08:49 |
Chris Johnston |
tags |
|
cts |
|
| 2015-02-12 00:08:57 |
Chris Johnston |
bug |
|
|
added subscriber Canonical Support |
| 2015-02-19 13:06:53 |
Edward Hope-Morley |
neutron: assignee |
|
Edward Hope-Morley (hopem) |
|
| 2015-02-19 13:08:24 |
Edward Hope-Morley |
affects |
neutron |
nova (Ubuntu) |
|
| 2015-02-19 13:08:24 |
Edward Hope-Morley |
nova (Ubuntu): assignee |
Edward Hope-Morley (hopem) |
|
|
| 2015-02-19 13:14:23 |
Edward Hope-Morley |
nova (Ubuntu): importance |
Undecided |
High |
|
| 2015-02-19 13:14:25 |
Edward Hope-Morley |
nova (Ubuntu): assignee |
|
Edward Hope-Morley (hopem) |
|
| 2015-02-19 13:14:27 |
Edward Hope-Morley |
nova (Ubuntu): status |
Confirmed |
In Progress |
|
| 2015-02-19 13:43:34 |
Edward Hope-Morley |
description |
There is a race when both neutron-ovs-cleanup and nova-compute trying to do operations on the qvb*** and qvo*** devices. Below is a scenario I recently met,
1. nova-compute was started and creating the veth_pair for VM instances running on the host - https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1298
2. neutron-ovs-cleanup was kicked off and deleted all the ports.
3. when nova-compute tried to set the MTU at https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1280 , Stderr: u'Cannot find device "qvo***"\n' was reported. Because the device that was just created was deleted again by neutron-ovs-cleanup.
As they both operate on the same resources, there needs a way to synchronize the operations the two processes do on those resources. |
[Impact]
* We run neutron-ovs-cleanup in startup if neutron installed. If nova-compute
does not wait for completion it will try to use veth/bridge devices that may
may be in the process of bring deleted.
[Test Case]
* Create neutron (ovs) network and boot an instance with this network as --nic
* Check that creation was successful and network is functional. Also make a
note corresponding veth and bridge devices (ip a).
* Reboot system, check that expected veth and bridge devices are still
there and that nova-compute is happy e.g. try sshing to your instance. Also
check /var/log/upstart/nova-compute.log to see if service waited for
ovs-cleanup to finish.
[Regression Potential]
* None
---- ---- ---- ----
There is a race when both neutron-ovs-cleanup and nova-compute trying to do operations on the qvb*** and qvo*** devices. Below is a scenario I recently met,
1. nova-compute was started and creating the veth_pair for VM instances running on the host - https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1298
2. neutron-ovs-cleanup was kicked off and deleted all the ports.
3. when nova-compute tried to set the MTU at https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1280 , Stderr: u'Cannot find device "qvo***"\n' was reported. Because the device that was just created was deleted again by neutron-ovs-cleanup.
As they both operate on the same resources, there needs a way to synchronize the operations the two processes do on those resources. |
|
| 2015-02-19 13:44:29 |
Edward Hope-Morley |
description |
[Impact]
* We run neutron-ovs-cleanup in startup if neutron installed. If nova-compute
does not wait for completion it will try to use veth/bridge devices that may
may be in the process of bring deleted.
[Test Case]
* Create neutron (ovs) network and boot an instance with this network as --nic
* Check that creation was successful and network is functional. Also make a
note corresponding veth and bridge devices (ip a).
* Reboot system, check that expected veth and bridge devices are still
there and that nova-compute is happy e.g. try sshing to your instance. Also
check /var/log/upstart/nova-compute.log to see if service waited for
ovs-cleanup to finish.
[Regression Potential]
* None
---- ---- ---- ----
There is a race when both neutron-ovs-cleanup and nova-compute trying to do operations on the qvb*** and qvo*** devices. Below is a scenario I recently met,
1. nova-compute was started and creating the veth_pair for VM instances running on the host - https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1298
2. neutron-ovs-cleanup was kicked off and deleted all the ports.
3. when nova-compute tried to set the MTU at https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1280 , Stderr: u'Cannot find device "qvo***"\n' was reported. Because the device that was just created was deleted again by neutron-ovs-cleanup.
As they both operate on the same resources, there needs a way to synchronize the operations the two processes do on those resources. |
[Impact]
* We run neutron-ovs-cleanup in startup if neutron installed. If
nova-compute does not wait for completion it will try to use
veth/bridge devices that may be in the process of bring deleted.
[Test Case]
* Create neutron (ovs) network and boot an instance with this network
as --nic
* Check that creation was successful and network is functional. Also make
a note corresponding veth and bridge devices (ip a).
* Reboot system, check that expected veth and bridge devices are still
there and that nova-compute is happy e.g. try sshing to your instance.
Also check /var/log/upstart/nova-compute.log to see if service waited
for ovs-cleanup to finish.
[Regression Potential]
* None
---- ---- ---- ----
There is a race when both neutron-ovs-cleanup and nova-compute trying to do operations on the qvb*** and qvo*** devices. Below is a scenario I recently met,
1. nova-compute was started and creating the veth_pair for VM instances running on the host - https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1298
2. neutron-ovs-cleanup was kicked off and deleted all the ports.
3. when nova-compute tried to set the MTU at https://github.com/openstack/nova/blob/stable/icehouse/nova/network/linux_net.py#L1280 , Stderr: u'Cannot find device "qvo***"\n' was reported. Because the device that was just created was deleted again by neutron-ovs-cleanup.
As they both operate on the same resources, there needs a way to synchronize the operations the two processes do on those resources. |
|
| 2015-02-19 13:47:34 |
Edward Hope-Morley |
attachment added |
|
nova-compute-2014.1-lp1420572.patch https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1420572/+attachment/4322607/+files/nova-compute-2014.1-lp1420572.patch |
|
| 2015-02-19 13:48:12 |
Edward Hope-Morley |
attachment removed |
nova-compute-2014.1-lp1420572.patch https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1420572/+attachment/4322607/+files/nova-compute-2014.1-lp1420572.patch |
|
|
| 2015-02-19 14:07:06 |
Edward Hope-Morley |
attachment added |
|
nova-compute-2014.1-lp1420572.patch https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1420572/+attachment/4322616/+files/nova-compute-2014.1-lp1420572.patch |
|
| 2015-02-19 14:22:36 |
James Page |
nominated for series |
|
Ubuntu Vivid |
|
| 2015-02-19 14:22:36 |
James Page |
bug task added |
|
nova (Ubuntu Vivid) |
|
| 2015-02-19 14:22:36 |
James Page |
nominated for series |
|
Ubuntu Utopic |
|
| 2015-02-19 14:22:36 |
James Page |
bug task added |
|
nova (Ubuntu Utopic) |
|
| 2015-02-19 14:22:36 |
James Page |
nominated for series |
|
Ubuntu Trusty |
|
| 2015-02-19 14:22:36 |
James Page |
bug task added |
|
nova (Ubuntu Trusty) |
|
| 2015-02-19 15:35:18 |
Edward Hope-Morley |
attachment added |
|
nova-compute-2014.2-lp1420572.patch https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1420572/+attachment/4322631/+files/nova-compute-2014.2-lp1420572.patch |
|
| 2015-02-19 15:35:34 |
Edward Hope-Morley |
nova (Ubuntu Utopic): status |
New |
In Progress |
|
| 2015-02-19 15:35:37 |
Edward Hope-Morley |
nova (Ubuntu Utopic): importance |
Undecided |
High |
|
| 2015-02-19 15:35:39 |
Edward Hope-Morley |
nova (Ubuntu Trusty): importance |
Undecided |
High |
|
| 2015-02-19 15:35:46 |
Edward Hope-Morley |
nova (Ubuntu Trusty): status |
New |
In Progress |
|
| 2015-02-19 15:35:49 |
Edward Hope-Morley |
nova (Ubuntu Utopic): assignee |
|
Edward Hope-Morley (hopem) |
|
| 2015-02-19 15:35:51 |
Edward Hope-Morley |
nova (Ubuntu Trusty): assignee |
|
Edward Hope-Morley (hopem) |
|
| 2015-02-19 15:47:39 |
Nobuto Murata |
bug |
|
|
added subscriber Nobuto Murata |
| 2015-02-19 16:17:03 |
Ubuntu Foundations Team Bug Bot |
tags |
cts |
cts patch |
|
| 2015-02-19 16:17:09 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2015-02-23 16:33:43 |
Edward Hope-Morley |
attachment added |
|
nova-compute-2015.1-lp1420572.patch https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1420572/+attachment/4325595/+files/nova-compute-2015.1-lp1420572.patch |
|
| 2015-02-24 10:53:09 |
James Page |
nova (Ubuntu Vivid): status |
In Progress |
Fix Committed |
|
| 2015-02-24 11:39:00 |
Launchpad Janitor |
nova (Ubuntu Vivid): status |
Fix Committed |
Fix Released |
|
| 2015-03-29 15:05:06 |
Launchpad Janitor |
branch linked |
|
lp:~corey.bryant/nova/2014.1.4-0ubuntu2 |
|
| 2015-03-29 15:27:07 |
Launchpad Janitor |
branch linked |
|
lp:~corey.bryant/nova/2014.2.2-0ubuntu2 |
|
| 2015-03-30 20:11:32 |
Corey Bryant |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2015-03-30 20:11:38 |
Corey Bryant |
summary |
race between neutron-ovs-cleanup and nova-compute |
[SRU] race between neutron-ovs-cleanup and nova-compute |
|
| 2015-04-01 02:49:24 |
Ryan Beisner |
bug |
|
|
added subscriber Ryan Beisner |
| 2015-04-01 15:54:15 |
Chris J Arges |
nova (Ubuntu Trusty): status |
In Progress |
Fix Committed |
|
| 2015-04-01 15:54:19 |
Chris J Arges |
bug |
|
|
added subscriber SRU Verification |
| 2015-04-01 15:54:21 |
Chris J Arges |
tags |
cts patch |
cts patch verification-needed |
|
| 2015-04-01 15:55:23 |
Chris J Arges |
nova (Ubuntu Utopic): status |
In Progress |
Fix Committed |
|
| 2015-04-03 15:32:20 |
Sebastien Bacher |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2015-04-08 11:46:28 |
Corey Bryant |
tags |
cts patch verification-needed |
cts patch verification-done |
|
| 2015-04-13 19:49:55 |
Chris J Arges |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2015-04-13 19:52:48 |
Launchpad Janitor |
nova (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
| 2015-04-15 20:13:06 |
Launchpad Janitor |
nova (Ubuntu Utopic): status |
Fix Committed |
Fix Released |
|
| 2016-09-22 16:04:33 |
Gustavo Randich |
bug |
|
|
added subscriber Gustavo Randich |
| 2016-09-22 16:24:09 |
Daniel |
bug |
|
|
added subscriber Daniel |
