Activity log for bug #1894843

Date Who What changed Old value New value Message
2020-09-08 13:13:51 ITD27M01 bug added bug
2020-09-08 13:14:09 ITD27M01 summary [dvr_snat] Router update deletes rfp interface from qrouter event when VM port is present on this host [dvr_snat] Router update deletes rfp interface from qrouter even when VM port is present on this host
2020-09-09 14:25:59 Slawek Kaplonski tags l3-dvr-backlog
2020-09-14 10:11:08 Slawek Kaplonski neutron: importance Undecided Medium
2020-10-15 18:01:19 Michael Eischer bug added subscriber Michael Eischer
2021-02-10 10:39:10 Hemanth Nakkina neutron: assignee Hemanth Nakkina (hemanth-n)
2021-02-12 07:08:20 Dominique Poulain bug added subscriber Dominique Poulain
2021-02-12 07:09:49 Dominique Poulain tags l3-dvr-backlog l3-dvr-backlog seg
2021-02-16 14:49:20 Edward Hope-Morley neutron: status New In Progress
2021-02-24 09:55:21 Edward Hope-Morley neutron: status In Progress Fix Released
2021-02-24 09:55:43 Edward Hope-Morley bug task added cloud-archive
2021-02-24 09:55:53 Edward Hope-Morley nominated for series cloud-archive/ussuri
2021-02-24 09:55:53 Edward Hope-Morley bug task added cloud-archive/ussuri
2021-02-24 09:55:53 Edward Hope-Morley nominated for series cloud-archive/victoria
2021-02-24 09:55:53 Edward Hope-Morley bug task added cloud-archive/victoria
2021-02-24 09:56:15 Edward Hope-Morley bug task added neutron (Ubuntu)
2021-02-24 09:56:23 Edward Hope-Morley nominated for series Ubuntu Focal
2021-02-24 09:56:23 Edward Hope-Morley bug task added neutron (Ubuntu Focal)
2021-02-24 09:56:23 Edward Hope-Morley nominated for series Ubuntu Groovy
2021-02-24 09:56:23 Edward Hope-Morley bug task added neutron (Ubuntu Groovy)
2021-02-24 09:56:23 Edward Hope-Morley nominated for series Ubuntu Hirsute
2021-02-24 09:56:23 Edward Hope-Morley bug task added neutron (Ubuntu Hirsute)
2021-02-24 10:13:35 Edward Hope-Morley attachment added lp1894843-victoria.debdiff https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1894843/+attachment/5466582/+files/lp1894843-victoria.debdiff
2021-02-24 10:14:06 Edward Hope-Morley attachment added lp1894843-ussuri.debdiff https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1894843/+attachment/5466583/+files/lp1894843-ussuri.debdiff
2021-02-24 10:25:54 Edward Hope-Morley description Hello, In the case of dvr_snat l3 agents are deployed on hypervisors there can be race condition. The agent creates snat namespaces on each scheduled host and removes them at second step. At this second step agent removes the rfp interface from qrouter even when there is VM with floating IP on the host. When VM is deployed at the time of second step we can lost external access to VMs floating IP. The issue can be reproduced by hand: 1. Create tenant network and router with external gateway 2. Create VM with floating ip 3. Ensure that VM on the hypervisor without snat-* namespace 4. Set the router to disabled state (openstack router set --disable <router>) 5. Set the router to enabled state (openstack router set --enabled <router>) 6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface. Environment: 1. Neutron with ML2 OVS plugin. 2. L3 agents in dvr_snat mode on each hypervisor 3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch [Impact] When neutron schedules snat namespaces it sometimes deletes the rfp interface from qrouter namespaces which breaks external network (fip) connectivity. The fix prevents this from happening. [Test Case] * deploy Openstack (Ussuri or above) with dvr_snat enabled in compute hosts. * ensure min. 2 compute hosts * create one ext network and one private network * add private subnet to router and ext as gateway * check which compute has the snat ns (ip netns| grep snat) * create a vm on each compute host * check that qrouter ns on both computes has rfp interface * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp * disable and re-enable router * openstack router set --disable <router>; openstack router set --enable <router> * check again * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp [Regression Potential] This patch is in fact restoring expected behaviour and is not expected to introduce any new regressions. ------------------------------------------------------------------------- Hello, In the case of dvr_snat l3 agents are deployed on hypervisors there can be race condition. The agent creates snat namespaces on each scheduled host and removes them at second step. At this second step agent removes the rfp interface from qrouter even when there is VM with floating IP on the host. When VM is deployed at the time of second step we can lost external access to VMs floating IP. The issue can be reproduced by hand: 1. Create tenant network and router with external gateway 2. Create VM with floating ip 3. Ensure that VM on the hypervisor without snat-* namespace 4. Set the router to disabled state (openstack router set --disable <router>) 5. Set the router to enabled state (openstack router set --enabled <router>) 6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface. Environment: 1. Neutron with ML2 OVS plugin. 2. L3 agents in dvr_snat mode on each hypervisor 3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch
2021-02-24 10:30:48 Edward Hope-Morley neutron (Ubuntu Hirsute): status New Fix Released
2021-02-24 10:43:35 Edward Hope-Morley attachment added lp1894843-groovy.debdiff https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1894843/+attachment/5466598/+files/lp1894843-groovy.debdiff
2021-02-24 17:34:09 Corey Bryant neutron (Ubuntu Hirsute): status Fix Released Triaged
2021-02-24 17:34:13 Corey Bryant neutron (Ubuntu Hirsute): importance Undecided Medium
2021-02-24 17:34:17 Corey Bryant neutron (Ubuntu Groovy): status New Triaged
2021-02-24 17:34:21 Corey Bryant neutron (Ubuntu Focal): status New Triaged
2021-02-24 17:34:25 Corey Bryant cloud-archive/victoria: status New Triaged
2021-02-24 17:34:32 Corey Bryant cloud-archive/ussuri: status New Triaged
2021-02-24 17:34:37 Corey Bryant neutron (Ubuntu Focal): importance Undecided Medium
2021-02-24 17:34:43 Corey Bryant cloud-archive/ussuri: importance Undecided Medium
2021-02-24 17:38:36 Corey Bryant cloud-archive/victoria: importance Undecided Medium
2021-02-24 17:38:44 Corey Bryant neutron (Ubuntu Groovy): importance Undecided Medium
2021-02-24 19:50:54 Corey Bryant bug added subscriber Ubuntu Stable Release Updates Team
2021-02-25 07:21:26 Launchpad Janitor neutron (Ubuntu Hirsute): status Triaged Fix Released
2021-02-27 20:51:46 Edward Hope-Morley description [Impact] When neutron schedules snat namespaces it sometimes deletes the rfp interface from qrouter namespaces which breaks external network (fip) connectivity. The fix prevents this from happening. [Test Case] * deploy Openstack (Ussuri or above) with dvr_snat enabled in compute hosts. * ensure min. 2 compute hosts * create one ext network and one private network * add private subnet to router and ext as gateway * check which compute has the snat ns (ip netns| grep snat) * create a vm on each compute host * check that qrouter ns on both computes has rfp interface * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp * disable and re-enable router * openstack router set --disable <router>; openstack router set --enable <router> * check again * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp [Regression Potential] This patch is in fact restoring expected behaviour and is not expected to introduce any new regressions. ------------------------------------------------------------------------- Hello, In the case of dvr_snat l3 agents are deployed on hypervisors there can be race condition. The agent creates snat namespaces on each scheduled host and removes them at second step. At this second step agent removes the rfp interface from qrouter even when there is VM with floating IP on the host. When VM is deployed at the time of second step we can lost external access to VMs floating IP. The issue can be reproduced by hand: 1. Create tenant network and router with external gateway 2. Create VM with floating ip 3. Ensure that VM on the hypervisor without snat-* namespace 4. Set the router to disabled state (openstack router set --disable <router>) 5. Set the router to enabled state (openstack router set --enabled <router>) 6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface. Environment: 1. Neutron with ML2 OVS plugin. 2. L3 agents in dvr_snat mode on each hypervisor 3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch [Impact] When neutron schedules snat namespaces it sometimes deletes the rfp interface from qrouter namespaces which breaks external network (fip) connectivity. The fix prevents this from happening. [Test Case]  * deploy Openstack (Ussuri or above) with dvr_snat enabled in compute hosts.  * ensure min. 2 compute hosts  * create one ext network and one private network  * add private subnet to router and ext as gateway  * check which compute has the snat ns (ip netns| grep snat)  * create a vm on each compute host  * check that qrouter ns on both computes has rfp interface  * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp  * disable and re-enable router  * openstack router set --disable <router>; openstack router set --enable <router>  * check again  * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp [Where problems could occur] This patch is in fact restoring expected behaviour and is not expected to introduce any new regressions. ------------------------------------------------------------------------- Hello, In the case of dvr_snat l3 agents are deployed on hypervisors there can be race condition. The agent creates snat namespaces on each scheduled host and removes them at second step. At this second step agent removes the rfp interface from qrouter even when there is VM with floating IP on the host. When VM is deployed at the time of second step we can lost external access to VMs floating IP. The issue can be reproduced by hand: 1. Create tenant network and router with external gateway 2. Create VM with floating ip 3. Ensure that VM on the hypervisor without snat-* namespace 4. Set the router to disabled state (openstack router set --disable <router>) 5. Set the router to enabled state (openstack router set --enabled <router>) 6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface. Environment: 1. Neutron with ML2 OVS plugin. 2. L3 agents in dvr_snat mode on each hypervisor 3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch
2021-03-08 08:59:26 Łukasz Zemczak neutron (Ubuntu Groovy): status Triaged Fix Committed
2021-03-08 08:59:30 Łukasz Zemczak bug added subscriber SRU Verification
2021-03-08 08:59:35 Łukasz Zemczak tags l3-dvr-backlog seg l3-dvr-backlog seg verification-needed verification-needed-groovy
2021-03-08 13:30:33 Corey Bryant cloud-archive: status Triaged Fix Committed
2021-03-08 13:30:57 Corey Bryant tags l3-dvr-backlog seg verification-needed verification-needed-groovy l3-dvr-backlog seg verification-needed verification-needed-groovy verification-victoria-needed
2021-03-08 13:51:14 Łukasz Zemczak neutron (Ubuntu Focal): status Triaged Fix Committed
2021-03-08 13:51:21 Łukasz Zemczak tags l3-dvr-backlog seg verification-needed verification-needed-groovy verification-victoria-needed l3-dvr-backlog seg verification-needed verification-needed-focal verification-needed-groovy verification-victoria-needed
2021-03-08 18:12:31 Corey Bryant cloud-archive/ussuri: status Triaged Fix Committed
2021-03-08 18:12:33 Corey Bryant tags l3-dvr-backlog seg verification-needed verification-needed-focal verification-needed-groovy verification-victoria-needed l3-dvr-backlog seg verification-needed verification-needed-focal verification-needed-groovy verification-ussuri-needed verification-victoria-needed
2021-03-12 03:36:23 Brett Milford bug added subscriber Brett Milford
2021-03-16 10:36:06 Hemanth Nakkina tags l3-dvr-backlog seg verification-needed verification-needed-focal verification-needed-groovy verification-ussuri-needed verification-victoria-needed l3-dvr-backlog seg verification-done-groovy verification-needed verification-needed-focal verification-ussuri-needed verification-victoria-needed
2021-03-16 11:44:51 Hemanth Nakkina tags l3-dvr-backlog seg verification-done-groovy verification-needed verification-needed-focal verification-ussuri-needed verification-victoria-needed l3-dvr-backlog seg verification-done-focal verification-done-groovy verification-needed verification-ussuri-needed verification-victoria-needed
2021-03-16 11:50:59 Hemanth Nakkina nominated for series Ubuntu Bionic
2021-03-16 11:50:59 Hemanth Nakkina bug task added neutron (Ubuntu Bionic)
2021-03-17 03:01:27 Hemanth Nakkina tags l3-dvr-backlog seg verification-done-focal verification-done-groovy verification-needed verification-ussuri-needed verification-victoria-needed l3-dvr-backlog seg verification-done-focal verification-done-groovy verification-needed verification-ussuri-done verification-victoria-done
2021-03-20 03:57:14 Hemanth Nakkina nominated for series cloud-archive/train
2021-03-20 03:57:14 Hemanth Nakkina bug task added cloud-archive/train
2021-03-20 03:57:14 Hemanth Nakkina nominated for series cloud-archive/rocky
2021-03-20 03:57:14 Hemanth Nakkina bug task added cloud-archive/rocky
2021-03-20 03:57:14 Hemanth Nakkina nominated for series cloud-archive/queens
2021-03-20 03:57:14 Hemanth Nakkina bug task added cloud-archive/queens
2021-03-20 03:57:14 Hemanth Nakkina nominated for series cloud-archive/stein
2021-03-20 03:57:14 Hemanth Nakkina bug task added cloud-archive/stein
2021-03-22 05:01:33 Hemanth Nakkina attachment added lp1894843_train.debdiff https://bugs.launchpad.net/cloud-archive/+bug/1894843/+attachment/5478919/+files/lp1894843_train.debdiff
2021-03-22 05:02:01 Hemanth Nakkina attachment added lp1894843_stein.debdiff https://bugs.launchpad.net/cloud-archive/+bug/1894843/+attachment/5478920/+files/lp1894843_stein.debdiff
2021-03-22 17:52:37 Corey Bryant tags l3-dvr-backlog seg verification-done-focal verification-done-groovy verification-needed verification-ussuri-done verification-victoria-done l3-dvr-backlog seg verification-done verification-done-focal verification-done-groovy verification-ussuri-done verification-victoria-done
2021-03-23 23:42:12 Brian Murray removed subscriber Ubuntu Stable Release Updates Team
2021-03-23 23:43:57 Launchpad Janitor neutron (Ubuntu Focal): status Fix Committed Fix Released
2021-03-23 23:52:18 Launchpad Janitor neutron (Ubuntu Groovy): status Fix Committed Fix Released
2021-03-24 04:54:35 Hemanth Nakkina attachment added lp1894843_rocky.debdiff https://bugs.launchpad.net/cloud-archive/+bug/1894843/+attachment/5480348/+files/lp1894843_rocky.debdiff
2021-03-24 04:54:59 Hemanth Nakkina attachment added lp1894843_bionic.debdiff https://bugs.launchpad.net/cloud-archive/+bug/1894843/+attachment/5480349/+files/lp1894843_bionic.debdiff
2021-03-29 14:02:41 Chris MacNaughton cloud-archive/victoria: status Fix Committed Fix Released
2021-03-30 03:05:30 Hemanth Nakkina tags l3-dvr-backlog seg verification-done verification-done-focal verification-done-groovy verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-ussuri-done verification-victoria-done
2021-04-05 20:45:59 Corey Bryant cloud-archive/train: status New Triaged
2021-04-05 20:46:44 Corey Bryant neutron (Ubuntu Bionic): importance Undecided High
2021-04-05 20:46:44 Corey Bryant neutron (Ubuntu Bionic): status New Triaged
2021-04-05 20:47:23 Corey Bryant cloud-archive/train: importance Undecided High
2021-04-05 20:47:37 Corey Bryant cloud-archive/stein: importance Undecided High
2021-04-05 20:47:37 Corey Bryant cloud-archive/stein: status New Triaged
2021-04-05 20:47:54 Corey Bryant cloud-archive/rocky: importance Undecided High
2021-04-05 20:47:54 Corey Bryant cloud-archive/rocky: status New Triaged
2021-04-05 20:48:06 Corey Bryant cloud-archive/queens: importance Undecided High
2021-04-05 20:48:06 Corey Bryant cloud-archive/queens: status New Triaged
2021-04-06 11:52:18 Corey Bryant cloud-archive/train: status Triaged Fix Committed
2021-04-06 11:52:21 Corey Bryant tags l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-train-needed verification-ussuri-done verification-victoria-done
2021-04-06 11:52:25 Corey Bryant cloud-archive/stein: status Triaged Fix Committed
2021-04-06 11:52:30 Corey Bryant cloud-archive/rocky: status Triaged Fix Committed
2021-04-06 13:53:47 Corey Bryant cloud-archive/ussuri: status Fix Committed Fix Released
2021-04-07 09:17:01 Hemanth Nakkina tags l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-train-needed verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-needed verification-stein-needed verification-train-needed verification-ussuri-done verification-victoria-done
2021-04-07 11:08:46 Hemanth Nakkina tags l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-needed verification-stein-needed verification-train-needed verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done
2021-04-12 13:49:34 Corey Bryant cloud-archive/train: status Fix Committed Fix Released
2021-04-12 13:50:32 Corey Bryant cloud-archive/stein: status Fix Committed Fix Released
2021-04-12 13:50:39 Corey Bryant cloud-archive/rocky: status Fix Committed Fix Released
2021-04-14 07:49:05 Chris MacNaughton cloud-archive/train: status Fix Released Fix Committed
2021-04-14 07:49:07 Chris MacNaughton tags l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-done verification-stein-done verification-train-needed verification-ussuri-done verification-victoria-done
2021-04-16 09:27:15 Hemanth Nakkina tags l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-done verification-stein-done verification-train-needed verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done
2021-05-04 15:56:51 Dan Streetman description [Impact] When neutron schedules snat namespaces it sometimes deletes the rfp interface from qrouter namespaces which breaks external network (fip) connectivity. The fix prevents this from happening. [Test Case]  * deploy Openstack (Ussuri or above) with dvr_snat enabled in compute hosts.  * ensure min. 2 compute hosts  * create one ext network and one private network  * add private subnet to router and ext as gateway  * check which compute has the snat ns (ip netns| grep snat)  * create a vm on each compute host  * check that qrouter ns on both computes has rfp interface  * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp  * disable and re-enable router  * openstack router set --disable <router>; openstack router set --enable <router>  * check again  * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp [Where problems could occur] This patch is in fact restoring expected behaviour and is not expected to introduce any new regressions. ------------------------------------------------------------------------- Hello, In the case of dvr_snat l3 agents are deployed on hypervisors there can be race condition. The agent creates snat namespaces on each scheduled host and removes them at second step. At this second step agent removes the rfp interface from qrouter even when there is VM with floating IP on the host. When VM is deployed at the time of second step we can lost external access to VMs floating IP. The issue can be reproduced by hand: 1. Create tenant network and router with external gateway 2. Create VM with floating ip 3. Ensure that VM on the hypervisor without snat-* namespace 4. Set the router to disabled state (openstack router set --disable <router>) 5. Set the router to enabled state (openstack router set --enabled <router>) 6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface. Environment: 1. Neutron with ML2 OVS plugin. 2. L3 agents in dvr_snat mode on each hypervisor 3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch [Impact] When neutron schedules snat namespaces it sometimes deletes the rfp interface from qrouter namespaces which breaks external network (fip) connectivity. The fix prevents this from happening. [Test Case]  * deploy Openstack (Ussuri or above) with dvr_snat enabled in compute hosts.  * ensure min. 2 compute hosts  * create one ext network and one private network  * add private subnet to router and ext as gateway  * check which compute has the snat ns (ip netns| grep snat)  * create a vm on each compute host  * check that qrouter ns on both computes has rfp interface  * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp  * disable and re-enable router  * openstack router set --disable <router>; openstack router set --enable <router>  * check again  * ip netns| grep qrouter; ip netns exec <ns> ip a s| grep rfp [Where problems could occur] no regression is expected, but if one occurs it would likely result in breakage with external network connectivity ------------------------------------------------------------------------- Hello, In the case of dvr_snat l3 agents are deployed on hypervisors there can be race condition. The agent creates snat namespaces on each scheduled host and removes them at second step. At this second step agent removes the rfp interface from qrouter even when there is VM with floating IP on the host. When VM is deployed at the time of second step we can lost external access to VMs floating IP. The issue can be reproduced by hand: 1. Create tenant network and router with external gateway 2. Create VM with floating ip 3. Ensure that VM on the hypervisor without snat-* namespace 4. Set the router to disabled state (openstack router set --disable <router>) 5. Set the router to enabled state (openstack router set --enabled <router>) 6. The external access to VMs FIP have lost because L3 agent creates the qrouter namespace without rfp interface. Environment: 1. Neutron with ML2 OVS plugin. 2. L3 agents in dvr_snat mode on each hypervisor 3. openstack-neutron-common-15.1.1-0.20200611111910.7d97420.el8ost.noarch
2021-05-05 09:45:13 Robie Basak neutron (Ubuntu Bionic): status Triaged Fix Committed
2021-05-05 09:45:16 Robie Basak bug added subscriber Ubuntu Stable Release Updates Team
2021-05-05 09:45:22 Robie Basak tags l3-dvr-backlog seg sts verification-done verification-done-focal verification-done-groovy verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done-focal verification-done-groovy verification-needed verification-needed-bionic verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done
2021-05-05 15:24:39 Corey Bryant cloud-archive/queens: status Triaged Fix Committed
2021-05-05 15:24:42 Corey Bryant tags l3-dvr-backlog seg sts verification-done-focal verification-done-groovy verification-needed verification-needed-bionic verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done-focal verification-done-groovy verification-needed verification-needed-bionic verification-queens-needed verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done
2021-05-06 06:46:12 Hemanth Nakkina tags l3-dvr-backlog seg sts verification-done-focal verification-done-groovy verification-needed verification-needed-bionic verification-queens-needed verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done-bionic verification-done-focal verification-done-groovy verification-needed verification-queens-needed verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done
2021-05-06 08:01:46 Hemanth Nakkina tags l3-dvr-backlog seg sts verification-done-bionic verification-done-focal verification-done-groovy verification-needed verification-queens-needed verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done l3-dvr-backlog seg sts verification-done verification-done-bionic verification-done-focal verification-done-groovy verification-queens-done verification-rocky-done verification-stein-done verification-train-done verification-ussuri-done verification-victoria-done
2021-05-10 12:26:17 Corey Bryant cloud-archive/train: status Fix Committed Fix Released
2021-05-13 08:03:40 Launchpad Janitor neutron (Ubuntu Bionic): status Fix Committed Fix Released
2021-05-13 15:04:17 Corey Bryant cloud-archive/queens: status Fix Committed Fix Released
2021-05-13 15:06:01 Corey Bryant cloud-archive: status Fix Committed Fix Released
2021-06-14 20:28:13 Paul Goins bug added subscriber alive-shad (BootStack)
2021-06-17 15:48:59 Paul Goins removed subscriber alive-shad (BootStack)
2021-10-12 12:00:52 Launchpad Janitor merge proposal linked https://code.launchpad.net/~hopem/ubuntu/+source/neutron/+git/neutron/+merge/410050
2021-10-12 12:01:14 Launchpad Janitor merge proposal linked https://code.launchpad.net/~hopem/ubuntu/+source/neutron/+git/neutron/+merge/410051
2021-10-12 12:01:22 Launchpad Janitor merge proposal linked https://code.launchpad.net/~hopem/ubuntu/+source/neutron/+git/neutron/+merge/410052