Invalid calls to ufw when cms-client-bound-address contains IPv6

[Simon Déziel]

The ovsdb-cms interface requires providing the CMS' IP address in cms-client-bound-address [1]. This address is surrounded by square brackets ("[]") when it is an IPv6 one [2].

When doing so, ufw is called with the square brackets left in place causing an error:

2022-03-22 19:24:43 DEBUG unit.ovn-central/1.juju-log server.go:327 ovsdb-cms:21: ufw allow: ufw prepend allow from [2602:fc62:b:3002:0:1:0:2] to any port 6641 proto tcp comment charm-ovn-central
2022-03-22 19:24:44 ERROR unit.ovn-central/1.juju-log server.go:327 ovsdb-cms:21: Error running: ufw prepend allow from [2602:fc62:b:3002:0:1:0:2] to any port 6641 proto tcp comment charm-ovn-central, exit code: 1
2022-03-22 19:24:44 DEBUG unit.ovn-central/1.juju-log server.go:327 ovsdb-cms:21: ufw allow: ufw prepend allow from [2602:fc62:b:3002:0:1:0:2] to any port 16642 proto tcp comment charm-ovn-central
2022-03-22 19:24:44 ERROR unit.ovn-central/1.juju-log server.go:327 ovsdb-cms:21: Error running: ufw prepend allow from [2602:fc62:b:3002:0:1:0:2] to any port 16642 proto tcp comment charm-ovn-central, exit code: 1

However, omitting the square brackets cause failures somewhere else preventing ufw from even being called.

1: https://opendev.org/x/charm-interface-ovsdb/src/branch/master/src/ovsdb_cms/requires.py#L40-L41
2: https://opendev.org/x/charm-interface-ovsdb/src/branch/master/src/lib/ovsdb.py#L33-L61

[Simon Déziel (sdeziel)]

While chatting with Frode Nordahl, he suggested the following quick fix (that I have not tested):

diff --git a/src/lib/charm/openstack/ovn_central.py b/src/lib/charm/openstack/ovn_central.py
index d9cd65b..f5ba614 100644
--- a/src/lib/charm/openstack/ovn_central.py
+++ b/src/lib/charm/openstack/ovn_central.py
@@ -642,6 +642,7 @@ class BaseOVNCentralCharm(charms_openstack.charm.OpenStackCharm):
             _addrs = list(addrs or [])
             for port in ports:
                 for addr in _addrs:
+ addr = addr.replace('[', '').replace(']', '')
                     ch_ufw.modify_access(addr, port=port, proto='tcp',
                                          action='allow', prepend=True,
                                          comment=ufw_comment)

[Frode Nordahl (fnordahl)]

Thank you for reporting this issue.

There are multiple facets to the solution:

1) We need to ensure the on the wire relation protocol uses pure IPv4/IPv6 address strings so that the remote end is free to compose the data in any shape/form on their discretion.

The both good and bad news here is that the current implementation is broken, so that we do not need to worry about breaking any in the wild users when fixing the on the wire protocol as no one would be able to successfully use the charm when bound to a IPv6 space at the moment.

2) The interface code is used to render the on the wire data for use in configuration templates as well as arguments to command execution, we may need separate properties for representing the pure ipv6 address string and the ipv6 address string encapsulated in brackets (`[]`).

3) Functional testing.

The current automatic test gates do not provide IPv6 connectivity and we do have a few dependencies on other projects in order to reliably enable that.

[Federico Bosi (rhxto)]

I'm having the same issue in rev 165.
Moreover, when we call ovsdb-cluster/lib/ovsdb.py@_remote_addrs when joining the cluster,
We get the remote addresses via addr = self.format_addr(unit.received.get(key, '')) (key is 'bound-address') which has the square brackets.
This results in the _remote_addrs function returning an empty array for the remote connection string (format_address raises a ValueError because of the brackets, and we *ignore* value errors), and joining the cluster fails as we provide no remote addresses:
unit-ovn-central-0: 08:36:03 INFO unit.ovn-central/0.juju-log ovsdb-peer:17: ['ovsdb-tool', 'join-cluster', '/var/lib/ovn/ovnnb_db.db', 'OVN_Northbound', 'ssl:[my_local_ipv6_addr]:6643']
unit-ovn-central-0: 08:36:03 ERROR unit.ovn-central/0.juju-log ovsdb-peer:17: Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.10/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-ovn-central-0/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-ovn-central-0/charm/reactive/ovn_central_handlers.py", line 245, in render
    ovn_charm.join_cluster('ovnnb_db.db', 'OVN_Northbound',
  File "/var/lib/juju/agents/unit-ovn-central-0/charm/lib/charm/openstack/ovn_central.py", line 500, in join_cluster
    self.run(*cmd)
  File "/var/lib/juju/agents/unit-ovn-central-0/charm/lib/charm/openstack/ovn_central.py", line 452, in run
    cp = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('ovsdb-tool', 'join-cluster', '/var/lib/ovn/ovnnb_db.db', 'OVN_Northbound', 'ssl:[my_local_ipv6_addr]:6643')' returned non-zero exit status 1.

The exit status is because we need to pass at least 4 arguments.
I just added a .replace("[", "").replace("]", "") to the format_address function which resolved the first issue and I'm now facing this one.
Both these issues share the same cause and I'd agree with Frode, ipv6 addresses should be passed as pure addresses and formatted when necessary.

[Federico Bosi (rhxto)]

ovn-chassis is affected by the same issue.
The local connection strings render successfully, while the remote ones are empty as _format_addr raises a ValueError which is ignored.
unit-ovn-chassis-2: 09:19:55 ERROR unit.ovn-chassis/2.juju-log ovsdb:82: Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ovn-chassis-2/.venv/lib/python3.10/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-ovn-chassis-2/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-ovn-chassis-2/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-ovn-chassis-2/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-ovn-chassis-2/charm/reactive/ovn_chassis_charm_handlers.py", line 133, in configure_ovs
    charm_instance.configure_ovs(
  File "/var/lib/juju/agents/unit-ovn-chassis-2/charm/lib/charms/ovn_charm.py", line 178, in configure_ovs
    super().configure_ovs(sb_conn, mlockall_changed)
  File "/var/lib/juju/agents/unit-ovn-chassis-2/charm/lib/charms/ovn_charm.py", line 1115, in configure_ovs
    self.run(*cmd)
  File "/var/lib/juju/agents/unit-ovn-chassis-2/charm/lib/charms/ovn_charm.py", line 863, in run
    cp = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('ovs-vsctl', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-type=geneve', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-ip=[local_ipv6_adddr]', '--', 'set', 'open-vswitch', '.', 'external-ids:system-id=giacomo.maas', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-remote=', '--', 'set', 'open-vswitch', '.', 'external_ids:ovn-match-northd-version=True')' returned non-zero exit status 1.

Note how the argument ovn-remote is empty: external-ids:ovn-remote=
Adding the same replace to format addr to ovn-chassis/charm/hooks/relations/ovsdb/lib/ovsdb.py fixes the issue, and now my ovn cluster is fully functional on ipv6.