Setting admin_state_up=False on an HA router with gateway raises an exception in standby nodes

[John Schwarz]

Steps to reproduce:

1. Create an HA router,
2. Connect the router to a gateway,
3. neutron router-update <router> --admin-state-down=False

This results in the following traceback on the standby l3 agents:

2015-10-12 14:43:44.755 ERROR neutron.agent.l3.router_info [-]
Command: ['ip', 'netns', 'exec', u'qrouter-0ce494ff-593a-4d6d-bf06-248979d6cf7a', 'ip', '-4', 'addr', 'del', '172.24.4.11/24', 'dev', u'qg-4f6a7587-00']
Exit code: 2
Stdin:
Stdout:
Stderr: RTNETLINK answers: Cannot assign requested address

2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info Traceback (most recent call last):
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/common/utils.py", line 356, in call
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info return func(*args, **kwargs)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/l3/router_info.py", line 695, in process
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info self.process_external(agent)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/l3/router_info.py", line 661, in process_external
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info self._process_external_gateway(ex_gw_port, agent.pd)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/l3/router_info.py", line 575, in _process_external_gateway
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info self.external_gateway_removed(self.ex_gw_port, interface_name)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/l3/ha_router.py", line 368, in external_gateway_removed
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info interface_name)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/l3/router_info.py", line 550, in external_gateway_removed
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info ip_addr['prefixlen']))
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/l3/router_info.py", line 201, in remove_external_gateway_ip
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info device.delete_addr_and_conntrack_state(ip_cidr)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/linux/ip_lib.py", line 255, in delete_addr_and_conntrack_state
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info self.addr.delete(cidr)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/linux/ip_lib.py", line 511, in delete
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info 'dev', self.name))
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/linux/ip_lib.py", line 295, in _as_root
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info use_root_namespace=use_root_namespace)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/linux/ip_lib.py", line 80, in _as_root
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/linux/ip_lib.py", line 89, in _execute
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info log_fail_as_error=log_fail_as_error)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info File "/opt/openstack/neutron/neutron/agent/linux/utils.py", line 157, in execute
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info raise RuntimeError(m)
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info RuntimeError:
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info Command: ['ip', 'netns', 'exec', u'qrouter-0ce494ff-593a-4d6d-bf06-248979d6cf7a', 'ip', '-4', 'addr', 'del', '172.24.4.11/24', 'dev', u'qg-4f6a7587-00']
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info Exit code: 2
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info Stdin:
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info Stdout:
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info Stderr: RTNETLINK answers: Cannot assign requested address
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info
2015-10-12 14:43:44.755 TRACE neutron.agent.l3.router_info
2015-10-12 14:43:44.755 ERROR neutron.agent.l3.agent [-] Error while deleting router 0ce494ff-593a-4d6d-bf06-248979d6cf7a

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/233617

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/233617
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b8f17bac76171993d6d276c51264fea8becb6136
Submitter: Jenkins
Branch: master

commit b8f17bac76171993d6d276c51264fea8becb6136
Author: John Schwarz <email address hidden>
Date: Mon Oct 12 15:53:49 2015 +0300

    Don't remove ip addresses if not master

    When setting --admin-state-up=False on an HA router with a gateway set,
    standby nodes don't have any ip addresses set on the devices (although
    the devices themselves are present). In such cases, Neutron should not
    try to un-set those ip addresses before deleting the device itself.

    Closes-bug: #1505203
    Change-Id: I5df04b2ed1dc08286f6c467111c61c7f97643d66

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/244054

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/244054
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=de1358d431a659366d61738f5a183825ad612f95
Submitter: Jenkins
Branch: stable/liberty

commit de1358d431a659366d61738f5a183825ad612f95
Author: John Schwarz <email address hidden>
Date: Mon Oct 12 15:53:49 2015 +0300

    Don't remove ip addresses if not master

    When setting --admin-state-up=False on an HA router with a gateway set,
    standby nodes don't have any ip addresses set on the devices (although
    the devices themselves are present). In such cases, Neutron should not
    try to un-set those ip addresses before deleting the device itself.

    Closes-bug: #1505203
    Change-Id: I5df04b2ed1dc08286f6c467111c61c7f97643d66
    (cherry picked from commit b8f17bac76171993d6d276c51264fea8becb6136)