Error while stoping radvd on dvr ha router

[Slawek Kaplonski]

In neutron-tempest-dvr-multinode-ha-full I found errors in L3 agent's logs. Error example is on:
http://logs.openstack.org/79/633979/11/check/neutron-tempest-dvr-ha-multinode-full/415f056/controller/logs/screen-q-l3.txt.gz?level=ERROR#_Feb_22_12_00_34_232612

Error looks like:

Feb 22 12:00:34.219810 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Exception: Unable to process HA router 1f90a750-d3af-454a-9031-29f399cf6457 without HA port
Feb 22 12:00:34.219810 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent [-] Error while deleting router 1f90a750-d3af-454a-9031-29f399cf6457: AttributeError: 'NoneType' object has no attribute 'disable'
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Traceback (most recent call last):
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 447, in _router_added
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent ri.delete()
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/ha_router.py", line 461, in delete
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent super(HaRouter, self).delete()
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 431, in delete
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent self.disable_radvd()
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 547, in disable_radvd
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent self.radvd.disable()
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'disable'
Feb 22 12:00:34.232612 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent [-] Failed to process compatible router: 1f90a750-d3af-454a-9031-29f399cf6457: Exception: Unable to process HA router 1f90a750-d3af-454a-9031-29f399cf6457 without HA port
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Traceback (most recent call last):
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 687, in _process_routers_if_compatible
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent self._process_router_if_compatible(router)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 562, in _process_router_if_compatible
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent self._process_added_router(router)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 567, in _process_added_router
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent self._router_added(router['id'], router)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 450, in _router_added
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent router_id)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent self.force_reraise()
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 439, in _router_added
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent ri.initialize(self.process_monitor)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/ha_router.py", line 126, in initialize
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent raise Exception(msg)
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent Exception: Unable to process HA router 1f90a750-d3af-454a-9031-29f399cf6457 without HA port
Feb 22 12:00:34.235509 ubuntu-bionic-rax-ord-0002911263 neutron-l3-agent[21802]: ERROR neutron.agent.l3.agent

Funny thing is that I found it on my patch https://review.openstack.org/#/c/633979/ which is migrating job definition to zuulv3 syntax. There is no such error in existing legacy job because it is not creates HA routers by default. So this is another issue that neutron-tempest-dvr-HA-multinode-full job is actually not HA but that can be fixed in https://review.openstack.org/#/c/633979/ after this issue reported here will be fixed.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/640006

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/640006
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fe4e7724cde61a036262a392abd1be76013f699c
Submitter: Zuul
Branch: master

commit fe4e7724cde61a036262a392abd1be76013f699c
Author: Slawek Kaplonski <email address hidden>
Date: Thu Feb 28 12:32:18 2019 +0100

    Don't disable radvd if radvd if it wasn't initialized

    In some cases on dvr ha router it may happend that
    RouterInfo.radvd.disable() will be called even if
    radvd DaemonMonitor wasn't initialized earlier and it is
    None.
    To prevent exception in such case, this patch adds check
    if DaemonMonitor is not None to call disable() method on
    it.

    Change-Id: Ib9b5f4eeae6e4cebcb958928e6521cf1d69b049c
    Closes-Bug: #1817435

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.0.0b3 development milestone.