Activity log for bug #1940043

Date Who What changed Old value New value Message
2021-08-16 08:56:51 Frode Nordahl bug added bug
2021-08-16 08:56:58 Frode Nordahl charm-ovn-chassis: status New In Progress
2021-08-16 08:57:02 Frode Nordahl charm-ovn-chassis: importance Undecided High
2021-08-16 08:57:05 Frode Nordahl charm-ovn-chassis: assignee Frode Nordahl (fnordahl)
2021-08-16 09:40:27 Frode Nordahl bug task added ovn (Ubuntu)
2021-08-16 09:43:37 Frode Nordahl description The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. The charms should enable this option to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413 The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413
2021-08-16 09:44:05 Frode Nordahl nominated for series Ubuntu Hirsute
2021-08-16 09:44:05 Frode Nordahl bug task added ovn (Ubuntu Hirsute)
2021-08-16 09:44:05 Frode Nordahl nominated for series Ubuntu Focal
2021-08-16 09:44:05 Frode Nordahl bug task added ovn (Ubuntu Focal)
2021-08-16 09:44:05 Frode Nordahl nominated for series Ubuntu Impish
2021-08-16 09:44:05 Frode Nordahl bug task added ovn (Ubuntu Impish)
2021-08-16 09:44:17 Frode Nordahl ovn (Ubuntu Impish): status New In Progress
2021-08-16 09:44:20 Frode Nordahl ovn (Ubuntu Impish): importance Undecided High
2021-08-16 09:44:23 Frode Nordahl ovn (Ubuntu Impish): assignee Frode Nordahl (fnordahl)
2021-08-16 09:56:33 Launchpad Janitor merge proposal linked https://code.launchpad.net/~fnordahl/ubuntu/+source/ovn/+git/ovn/+merge/407170
2021-08-16 09:57:40 Frode Nordahl bug task added charm-layer-ovn
2021-08-16 09:57:45 Frode Nordahl charm-layer-ovn: status New Incomplete
2021-08-16 09:57:50 Frode Nordahl charm-layer-ovn: status Incomplete In Progress
2021-08-16 09:57:53 Frode Nordahl charm-layer-ovn: importance Undecided High
2021-08-16 09:57:56 Frode Nordahl charm-layer-ovn: assignee Frode Nordahl (fnordahl)
2021-08-16 09:58:08 Frode Nordahl bug task added charm-ovn-dedicated-chassis
2021-08-16 09:58:15 Frode Nordahl charm-ovn-dedicated-chassis: status New In Progress
2021-08-16 09:58:18 Frode Nordahl charm-ovn-dedicated-chassis: importance Undecided High
2021-08-16 09:58:22 Frode Nordahl charm-ovn-dedicated-chassis: assignee Frode Nordahl (fnordahl)
2021-08-16 09:58:27 Frode Nordahl charm-ovn-dedicated-chassis: status In Progress Triaged
2021-08-16 09:58:31 Frode Nordahl charm-ovn-chassis: status In Progress Triaged
2021-08-16 13:24:50 Aurelien Lourot charm-layer-ovn: status In Progress Fix Committed
2021-08-16 13:24:54 Aurelien Lourot charm-layer-ovn: milestone 21.10
2021-08-16 13:27:14 OpenStack Infra charm-ovn-chassis: status Triaged In Progress
2021-08-16 13:28:17 OpenStack Infra charm-ovn-dedicated-chassis: status Triaged In Progress
2021-08-16 16:55:32 OpenStack Infra charm-ovn-dedicated-chassis: status In Progress Fix Committed
2021-08-16 16:56:32 OpenStack Infra charm-ovn-chassis: status In Progress Fix Committed
2021-08-24 06:21:52 Frode Nordahl ovn (Ubuntu Impish): status In Progress Fix Committed
2021-08-24 06:23:35 Frode Nordahl ovn (Ubuntu Impish): assignee Frode Nordahl (fnordahl)
2021-08-24 06:23:39 Frode Nordahl ovn (Ubuntu Hirsute): assignee Frode Nordahl (fnordahl)
2021-08-24 06:23:41 Frode Nordahl ovn (Ubuntu Focal): assignee Frode Nordahl (fnordahl)
2021-09-04 02:23:13 Launchpad Janitor ovn (Ubuntu Impish): status Fix Committed Fix Released
2021-10-11 15:18:10 Alex Kavanagh charm-ovn-chassis: milestone 21.10
2021-10-11 15:18:12 Alex Kavanagh charm-ovn-dedicated-chassis: milestone 21.10
2021-10-22 13:24:34 Alex Kavanagh charm-ovn-chassis: status Fix Committed Fix Released
2021-10-22 13:24:36 Alex Kavanagh charm-layer-ovn: status Fix Committed Fix Released
2021-10-22 13:24:38 Alex Kavanagh charm-ovn-dedicated-chassis: status Fix Committed Fix Released
2022-01-26 22:01:56 Brian Murray ovn (Ubuntu Hirsute): status New Won't Fix
2022-03-22 10:02:10 Frode Nordahl bug task added cloud-archive
2022-03-22 10:04:43 Frode Nordahl nominated for series cloud-archive/wallaby
2022-03-22 10:04:43 Frode Nordahl bug task added cloud-archive/wallaby
2022-03-22 10:04:52 Frode Nordahl cloud-archive: status New Fix Released
2022-03-22 10:05:01 Frode Nordahl cloud-archive/wallaby: importance Undecided High
2022-03-22 10:05:04 Frode Nordahl cloud-archive/wallaby: assignee Frode Nordahl (fnordahl)
2022-03-22 10:05:43 Frode Nordahl cloud-archive/wallaby: status New Triaged
2022-03-22 10:05:46 Frode Nordahl ovn (Ubuntu Focal): status New Triaged
2022-07-04 13:47:59 Launchpad Janitor merge proposal linked https://code.launchpad.net/~fnordahl/ubuntu/+source/ovn/+git/ovn/+merge/426199
2022-07-04 14:03:08 Frode Nordahl description The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413 [Impact] When upgrading from OVN 20.03, as made available in Ubuntu Focal, to a newer version of OVN, it is currently not possible to upgrade the central components first. Doing so will make the ovn-controller tear down connectivity to running instances as it may not fully understand the data structure of a newer database. To fix this situation we have backported a upstream feature [0] that allows the ovn-controller to detect version mismatch and subsequently refrain from making further changes to the local Open vSwitch instance until the version mismatch is corrected. [Test Plan] 1. Deploy OpenStack Ussuri from the Focal archive. 2. Launch and instance and confirm connectivity. 3. Add UCA or other PPA with a newer version of OVN and perform upgrade of the OVN components on relevant units in the deployment. 4. Confirm how new version of central components make the ovn-controller log version mismatch as well as show continued connectivity to the test instance. 5. Upgrade data plane units and confirm how the version mismatch situation is resolved and at the same time instances retain connectivity with minimal downtime during the upgrade. [Regression Potential] The backported feature is optional and enabled by specifically entering a key-value pair into the local Open vSwitch database to enable it. It has also been available upstream for several releases. [Original Bug Description] The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413
2022-07-29 08:18:49 Frode Nordahl bug added subscriber Ubuntu Stable Release Updates Team
2022-08-05 13:19:05 Frode Nordahl ovn (Ubuntu Focal): importance Undecided High
2022-08-05 21:29:37 Steve Langasek ovn (Ubuntu Focal): status Triaged Incomplete
2022-08-06 05:35:07 Frode Nordahl description [Impact] When upgrading from OVN 20.03, as made available in Ubuntu Focal, to a newer version of OVN, it is currently not possible to upgrade the central components first. Doing so will make the ovn-controller tear down connectivity to running instances as it may not fully understand the data structure of a newer database. To fix this situation we have backported a upstream feature [0] that allows the ovn-controller to detect version mismatch and subsequently refrain from making further changes to the local Open vSwitch instance until the version mismatch is corrected. [Test Plan] 1. Deploy OpenStack Ussuri from the Focal archive. 2. Launch and instance and confirm connectivity. 3. Add UCA or other PPA with a newer version of OVN and perform upgrade of the OVN components on relevant units in the deployment. 4. Confirm how new version of central components make the ovn-controller log version mismatch as well as show continued connectivity to the test instance. 5. Upgrade data plane units and confirm how the version mismatch situation is resolved and at the same time instances retain connectivity with minimal downtime during the upgrade. [Regression Potential] The backported feature is optional and enabled by specifically entering a key-value pair into the local Open vSwitch database to enable it. It has also been available upstream for several releases. [Original Bug Description] The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413 [Impact] When upgrading from OVN 20.03, as made available in Ubuntu Focal, to a newer version of OVN, it is currently not possible to upgrade the central components first. Doing so will make the ovn-controller tear down connectivity to running instances as it may not fully understand the data structure of a newer database. To fix this situation we have backported a upstream feature [0] that allows the ovn-controller to detect version mismatch and subsequently refrain from making further changes to the local Open vSwitch instance until the version mismatch is corrected. In order to minimize the downtime on package upgrade, and to cater for anyone both enabling the version mismatch feature and upgrading the controller first, the ovn-controller systemd service is also updated to pass the `--restart` argument when stopping the controller. This flag tells the ovn-controller process that it should not clear out Open vSwitch flows and OVN SB database records on exit, which allows already installed state to continue operation until the new instance of the ovn-controller process starts. [1][2][3] [Test Plan] 1. Deploy OpenStack Ussuri from the Focal archive. 2. Launch and instance and confirm connectivity. 3. Add UCA or other PPA with a newer version of OVN and perform upgrade of the OVN components on relevant units in the deployment. 4. Confirm how new version of central components make the ovn-controller log version mismatch as well as show continued connectivity to the test instance. 5. Upgrade data plane units and confirm how the version mismatch situation is resolved and at the same time instances retain connectivity with minimal downtime during the upgrade. [Regression Potential] The backported feature is optional and enabled by specifically entering a key-value pair into the local Open vSwitch database to enable it. It has also been available upstream for several releases. The change to the ovn-controller systemd service has been in Ubuntu since Impish [3] and we have had no reports of side effects of this change. [Original Bug Description] The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413 1: https://github.com/ovn-org/ovn/commit/f508fcc14abfaaa13e9f1bf3b5b6bac59bd27a5f 2: https://github.com/ovn-org/ovn/commit/45c7a85dc7f2af56191a47f1357d16b8af618e20 3: https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/ovn/commit/debian/ovn-host.ovn-controller.service?id=3c601ecc13724d3f13ec0cc989f6ffd838f787f8
2022-08-06 05:36:48 Frode Nordahl ovn (Ubuntu Focal): status Incomplete Triaged
2022-10-13 06:29:53 Frode Nordahl description [Impact] When upgrading from OVN 20.03, as made available in Ubuntu Focal, to a newer version of OVN, it is currently not possible to upgrade the central components first. Doing so will make the ovn-controller tear down connectivity to running instances as it may not fully understand the data structure of a newer database. To fix this situation we have backported a upstream feature [0] that allows the ovn-controller to detect version mismatch and subsequently refrain from making further changes to the local Open vSwitch instance until the version mismatch is corrected. In order to minimize the downtime on package upgrade, and to cater for anyone both enabling the version mismatch feature and upgrading the controller first, the ovn-controller systemd service is also updated to pass the `--restart` argument when stopping the controller. This flag tells the ovn-controller process that it should not clear out Open vSwitch flows and OVN SB database records on exit, which allows already installed state to continue operation until the new instance of the ovn-controller process starts. [1][2][3] [Test Plan] 1. Deploy OpenStack Ussuri from the Focal archive. 2. Launch and instance and confirm connectivity. 3. Add UCA or other PPA with a newer version of OVN and perform upgrade of the OVN components on relevant units in the deployment. 4. Confirm how new version of central components make the ovn-controller log version mismatch as well as show continued connectivity to the test instance. 5. Upgrade data plane units and confirm how the version mismatch situation is resolved and at the same time instances retain connectivity with minimal downtime during the upgrade. [Regression Potential] The backported feature is optional and enabled by specifically entering a key-value pair into the local Open vSwitch database to enable it. It has also been available upstream for several releases. The change to the ovn-controller systemd service has been in Ubuntu since Impish [3] and we have had no reports of side effects of this change. [Original Bug Description] The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413 1: https://github.com/ovn-org/ovn/commit/f508fcc14abfaaa13e9f1bf3b5b6bac59bd27a5f 2: https://github.com/ovn-org/ovn/commit/45c7a85dc7f2af56191a47f1357d16b8af618e20 3: https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/ovn/commit/debian/ovn-host.ovn-controller.service?id=3c601ecc13724d3f13ec0cc989f6ffd838f787f8 [Impact] When upgrading from OVN 20.03, as made available in Ubuntu Focal, to a newer version of OVN, it is currently not possible to upgrade without causing a data plane outage. If the user attempts to upgrade the central components first, the ovn-controller will tear down connectivity to running instances as it may not fully understand the data structure of a newer database. If the user attempts to upgrade the ovn-controler first, recent releases are not guaranteed to understand the older database and connectivity may remain down until all hypervisors and central components have been upgraded. If the user attempts to manually stop the ovn-controller during the upgrade to avoid it inadvertently tearing down connectivity on central component upgrade, cloud instances will be deprived of vital services such as DNS lookup and DHCP. To fix this situation two changes are needed: 1) Backport of a upstream feature [0] that allows the ovn-controller to detect version mismatch and subsequently refrain from making further changes to the local Open vSwitch instance until the version mismatch is corrected. 2) Make ovn-controller not clear out runtime flow state in Open vSwitch on exit by updating the ovn-controller systemd service to pass the `--restart` argument when stopping the controller. This flag tells the ovn-controller process that it should not clear out Open vSwitch flows and OVN SB database records on exit, which allows already installed state to continue operation until the new instance of the ovn-controller process starts. [1][2][3] It does not mean that the service will be restarted as opposed to being stopped, as one might think based on the name of the argument. This change serves two purposes: 2a) Allow upgrading the ovn-controller to a newer version than the central components, while retaining connectivity to running instances until the central components are upgraded. 2b) Minimize the downtime on package upgrade. [Test Plan] 1. Deploy OpenStack Ussuri from the Focal archive. 2. Launch and instance and confirm connectivity. 3. Add UCA or other PPA with a newer version of OVN and perform upgrade of the OVN components on relevant units in the deployment. 4. Confirm how new version of central components make the ovn-controller log version mismatch as well as show continued connectivity to the test instance. 5. Upgrade data plane units and confirm how the version mismatch situation is resolved and at the same time instances retain connectivity with minimal downtime during the upgrade. [Regression Potential] The backported feature is optional and enabled by specifically entering a key-value pair into the local Open vSwitch database to enable it. It has also been available upstream for several releases. The change to the ovn-controller systemd service has been in Ubuntu since Impish [3] and we have had no reports of side effects of this change. [Original Bug Description] The upstream recommendation for upgrades of OVN is to first upgrade the data plane components (chassis aka. ovn-controller), and then upgrade the central components (the database schema and ovn-northd). The rationale for this is that the new version of the ovn-controller is required to cope with any changes to database schema or how northd programs flows. However, during the course of rapid OVN development there has also been introduced changes that make the new ovn-controller not cope with a old database schema, breaking the recommended upgrade procedure. To cope with this upstream has introduced a new optional configuration for the ovn-controller that allows it to detect version inconsistencies, and when they are present stop it from making changes to the data plane until the version inconsistency is resolved [0]. For the above mentioned configuration to be effective we also need the package to call ``ovn-ctl stop_controller`` with the --restart option so that the ovn-controller does not flush the installed flows on exit. We should make required changes to packages and charms to allow upgrades to progress with less data plane outage. 0: https://github.com/ovn-org/ovn/commit/1dd27ea7aea40122c1edbff845e14abaa70c0413 1: https://github.com/ovn-org/ovn/commit/f508fcc14abfaaa13e9f1bf3b5b6bac59bd27a5f 2: https://github.com/ovn-org/ovn/commit/45c7a85dc7f2af56191a47f1357d16b8af618e20 3: https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/ovn/commit/debian/ovn-host.ovn-controller.service?id=3c601ecc13724d3f13ec0cc989f6ffd838f787f8
2022-10-21 09:50:24 Timo Aaltonen ovn (Ubuntu Focal): status Triaged Fix Committed
2022-10-21 09:50:28 Timo Aaltonen bug added subscriber SRU Verification
2022-10-21 09:50:30 Timo Aaltonen tags openstack-upgrade openstack-upgrade verification-needed verification-needed-focal
2022-11-06 12:08:08 Frode Nordahl tags openstack-upgrade verification-needed verification-needed-focal openstack-upgrade verification-done verification-done-focal
2023-03-01 00:37:43 Chris Halse Rogers removed subscriber Ubuntu Stable Release Updates Team
2023-03-01 00:59:36 Launchpad Janitor ovn (Ubuntu Focal): status Fix Committed Fix Released
2023-03-10 14:03:23 James Troup bug added subscriber James Troup
2023-03-21 09:15:58 James Page cloud-archive: status Fix Released Fix Committed
2023-03-21 15:28:49 James Page cloud-archive: status Fix Committed Fix Released
2023-06-27 21:48:13 Billy Olsen nominated for series charm-ovn-chassis/20.12
2023-06-27 21:48:13 Billy Olsen bug task added charm-ovn-chassis/20.12
2023-06-27 21:48:13 Billy Olsen nominated for series charm-ovn-chassis/21.09
2023-06-27 21:48:13 Billy Olsen bug task added charm-ovn-chassis/21.09
2023-06-27 21:48:13 Billy Olsen nominated for series charm-ovn-chassis/20.03
2023-06-27 21:48:13 Billy Olsen bug task added charm-ovn-chassis/20.03
2023-06-27 21:48:26 Billy Olsen charm-ovn-chassis/20.03: status New Fix Released
2023-06-27 21:48:34 Billy Olsen charm-ovn-chassis/20.12: status New Fix Released
2023-06-27 21:48:42 Billy Olsen charm-ovn-chassis/21.09: status New Fix Released
2023-06-27 21:49:05 Billy Olsen nominated for series charm-ovn-dedicated-chassis/20.03
2023-06-27 21:49:05 Billy Olsen bug task added charm-ovn-dedicated-chassis/20.03
2023-06-27 21:49:05 Billy Olsen nominated for series charm-ovn-dedicated-chassis/20.12
2023-06-27 21:49:05 Billy Olsen bug task added charm-ovn-dedicated-chassis/20.12
2023-06-27 21:49:05 Billy Olsen nominated for series charm-ovn-dedicated-chassis/21.09
2023-06-27 21:49:05 Billy Olsen bug task added charm-ovn-dedicated-chassis/21.09
2023-06-27 21:49:13 Billy Olsen charm-ovn-dedicated-chassis/20.03: status New Fix Released
2023-06-27 21:49:20 Billy Olsen charm-ovn-dedicated-chassis/20.12: status New Fix Released
2023-06-27 21:49:27 Billy Olsen charm-ovn-dedicated-chassis/21.09: status New Fix Released
2023-10-26 11:05:39 Jorge Rodríguez bug added subscriber Jorge Rodríguez