charm-ovn-chassis

ovsdb-relation-departed hook fails on ovn-chassis unit removal

Bug #1944983 reported by Vladimir Grevtsev
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
charm-ovn-chassis
In Progress
Medium
Felipe Reyes

Bug Description

== Env
focal/ussuri + ovn
nova-compute-dpdk 21.2.1 active 4 nova-compute charmstore stable 334 ubuntu Unit is ready
ovn-chassis-dpdk 20.03.2 error 4 ovn-chassis charmstore stable 17 ubuntu hook failed: "ovsdb-relation-departed"

nova-compute/ovn-chassis overlay: https://pastebin.canonical.com/p/2kBXYmdzqt/

== Problem
Unit falls into the error state with the following traceback in the unit logs:

unit-ovn-chassis-dpdk-0: 12:36:58 DEBUG unit.ovn-chassis-dpdk/0.juju-log ovsdb:205: Changing permissions on existing content: 33184 -> 416
unit-ovn-chassis-dpdk-0: 12:36:58 INFO unit.ovn-chassis-dpdk/0.juju-log ovsdb:205: CompletedProcess(args=('ovs-vsctl', '--no-wait', 'set-ssl', '/etc/ovn/key_host', '/etc/ovn/cert_host', '/etc/ovn/ovn-chassis.crt'), returncode=0, stdout='')
unit-ovn-chassis-dpdk-0: 12:36:58 ERROR unit.ovn-chassis-dpdk/0.juju-log ovsdb:205: Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/reactive/ovn_chassis_charm_handlers.py", line 114, in configure_ovs
    charm_instance.configure_ovs(
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 142, in configure_ovs
    super().configure_ovs(sb_conn, mlockall_changed)
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 747, in configure_ovs
    self.run(*cmd)
  File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 546, in run
    cp = subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('ovs-vsctl', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-type=geneve', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-ip=10.35.83.3', '--', 'set', 'open-vswitch', '.', 'external-ids:system-id=u0400s1entcomp02.fqdn', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-remote=', '--', 'set', 'open-vswitch', '.', 'external_ids:ovn-match-northd-version=true')' returned non-zero exit status 1.

unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed Traceback (most recent call last):
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/hooks/ovsdb-relation-departed", line 22, in <module>
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed main()
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed bus.dispatch(restricted=restricted_mode)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed _invoke(other_handlers)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed handler.invoke()
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed self._action(*args)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/reactive/ovn_chassis_charm_handlers.py", line 114, in configure_ovs
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed charm_instance.configure_ovs(
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 142, in configure_ovs
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed super().configure_ovs(sb_conn, mlockall_changed)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 747, in configure_ovs
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed self.run(*cmd)
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/var/lib/juju/agents/unit-ovn-chassis-dpdk-0/charm/lib/charms/ovn_charm.py", line 546, in run
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed cp = subprocess.run(
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed File "/usr/lib/python3.8/subprocess.py", line 516, in run
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed raise CalledProcessError(retcode, process.args,
unit-ovn-chassis-dpdk-0: 12:36:58 WARNING unit.ovn-chassis-dpdk/0.ovsdb-relation-departed subprocess.CalledProcessError: Command '('ovs-vsctl', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-type=geneve', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-encap-ip=10.35.83.3', '--', 'set', 'open-vswitch', '.', 'external-ids:system-id=u0400s1entcomp02.fqdn', '--', 'set', 'open-vswitch', '.', 'external-ids:ovn-remote=', '--', 'set', 'open-vswitch', '.', 'external_ids:ovn-match-northd-version=true')' returned non-zero exit status 1.
unit-ovn-chassis-dpdk-0: 12:36:58 ERROR juju.worker.uniter.operation hook "ovsdb-relation-departed" (via explicit, bespoke hook script) failed: exit status 1
unit-ovn-chassis-dpdk-0: 12:36:58 DEBUG juju.machinelock machine lock released for ovn-chassis-dpdk/0 uniter (run relation-departed (205; unit: ovn-central/2, departee: ovn-chassis-dpdk/0) hook)

Trying to run the above command manually:

# ovs-vsctl -- set open-vswitch . external-ids:ovn-encap-type=geneve -- set open-vswitch . external-ids:ovn-encap-ip=10.35.83.3 -- set open-vswitch . external-ids:system-id=u0400s1entcomp02.fqdn -- set open-vswitch . external-ids:ovn-remote= -- set open-vswitch . external_ids:ovn-match-northd-version=true
ovs-vsctl: external-ids:ovn-remote=: argument does not end in "=" followed by a value.

Tags: seg
Hemanth Nakkina (hemanth-n)
Changed in charm-ovn-chassis:
assignee: nobody → Hemanth Nakkina (hemanth-n)
Hemanth Nakkina (hemanth-n)
tags: added: seg
Revision history for this message
Felipe Reyes (freyes) wrote :

https://github.com/openstack-charmers/charm-layer-ovn/pull/63

Changed in charm-ovn-chassis:
status: New → In Progress
assignee: Hemanth Nakkina (hemanth-n) → Felipe Reyes (freyes)
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.