Clustered OVN database is not upgraded on package upgrade
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
charm-ovn-central |
Fix Released
|
Medium
|
Frode Nordahl | ||
ovn (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Focal |
Fix Released
|
Medium
|
Frode Nordahl | ||
Groovy |
Fix Released
|
Medium
|
Frode Nordahl | ||
Hirsute |
Fix Released
|
Medium
|
Unassigned |
Bug Description
[Impact]
On upgrade of the OVN packages it may be necessary to perform a upgrade to the Northbound and Southbound databases.
Failure to do so may lead to loss of connectivity between participating nodes as the software components will attempt to make use of columns that are not available in the database.
The upgrade process has been performed automatically by the upstream init script by default since inception, both for a local and clustered setup. But as discussed below recent changes has inadvertently omitted this behavior for clustered databases.
[Test Case]
Non-clustered scenario as reference test:
Install the ovn-central package in a container using the in-release focal package and start the database and ovn-northd services.
Upgrade the container to the OVN packages from in-release Groovy and observe the package performing the database upgrade and subsequently ovn-northd service not complaining about missing columns in the database.
Clustered scenario:
Install the ovn-central charm across three containers and necessary dependencies. Perform package upgrade as outlined above and compare how in-relase and proposed packages behave.
[Regression Potential]
As we are restoring the intended behavior the regression potential is minimal.
[Original Bug Report]
In the systemd service we make use of the `ovn-ctl` script `run_nb_ovsdb` and `run_sb_ovsdb` sub-commands introduced in [0]. These sub-commands fit nicely with systemd's expectations of modern daemons to no longer detachand run in the background.
However, the change in [0] has the side effect of disabling automatic upgrading of clustered databases. Previously this would have been done on every startup [1].
A recent commit to master [2] addresses this and uses the combination of presence of `--db-*
We should backport [2] to our supported OVN packages to prepare for supporting upgrades that require database schema changes. We may also need to change the behavior of the ovn-central charm to not set the `--db-*
0: https:/
1: https:/
2: https:/
Related branches
- Ubuntu Server Developers: Pending requested
-
Diff: 4356 lines (+4230/-0) (has conflicts)20 files modifieddebian/changelog (+16/-0)
debian/patches/ovn-ctl-cluster-db-upgrades.patch (+63/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-01.patch (+63/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-02.patch (+889/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-03.patch (+718/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-04.patch (+154/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-05.patch (+481/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-06.patch (+109/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-07.patch (+233/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-08.patch (+90/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-09.patch (+216/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-10.patch (+77/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-11.patch (+215/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-12.patch (+40/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-13.patch (+416/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-14.patch (+123/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-15.patch (+66/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-16.patch (+132/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-17.patch (+108/-0)
debian/patches/series (+21/-0)
- Ubuntu Server Developers: Pending requested
-
Diff: 4592 lines (+4464/-0)21 files modifieddebian/changelog (+12/-0)
debian/patches/ovn-ctl-cluster-db-upgrades.patch (+63/-0)
debian/patches/ovn-northd-revert-manage-arp-process-locally-dvr.patch (+198/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-01.patch (+66/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-02.patch (+892/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-03.patch (+719/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-04.patch (+157/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-05.patch (+484/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-06.patch (+112/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-07.patch (+236/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-08.patch (+93/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-09.patch (+220/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-10.patch (+80/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-11.patch (+218/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-12.patch (+43/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-13.patch (+419/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-14.patch (+126/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-15.patch (+66/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-16.patch (+135/-0)
debian/patches/ovn-ofctrl-predictable-resolution-conflicting-flow-actions-17.patch (+106/-0)
debian/patches/series (+19/-0)
Changed in ovn (Ubuntu): | |
status: | New → Triaged |
importance: | Undecided → Medium |
Changed in charm-ovn-central: | |
status: | New → Triaged |
importance: | Undecided → Medium |
Changed in charm-ovn-central: | |
status: | Triaged → In Progress |
assignee: | nobody → Frode Nordahl (fnordahl) |
summary: |
- OVN database is not upgraded on package upgrade + Clustered OVN database is not upgraded on package upgrade |
Changed in ovn (Ubuntu): | |
assignee: | nobody → Frode Nordahl (fnordahl) |
Changed in ovn (Ubuntu): | |
status: | Triaged → In Progress |
description: | updated |
description: | updated |
description: | updated |
Changed in charm-ovn-central: | |
status: | Fix Committed → Fix Released |
https:/ /review. opendev. org/c/x/ charm-ovn- central/ +/765794