neutron

IPv6 prefix delegation does not work with DVR

Bug #1541406 reported by Brian Haley on 2016-02-03
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
​ neutron
Fix Released
Medium
Brian Haley
neutron ocata-rc1 "rc1"
Also affects project (?) Also affects distribution/package Nominate for series

Bug Description

Using a recent single-node devstack install, I tried to enable IPv6 prefix delegation following the exact instructions in the advanced networking guide - http://docs.openstack.org/liberty/networking-guide/adv_config_ipv6.html

$ neutron net-create ipv6-pd
$ neutron subnet-create ipv6-pd --name ipv6-pd-1 --ip_version 6 --ipv6_ra_mode slaac --ipv6_address_mode slaac
$ neutron router-interface-add ...

The l3-agent threw an exception:

2016-02-02 10:28:07.014 23328 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-f932415c-2cbd-4bcf-a54e-d7f7ee890908', 'ip', '-6', 'addr', 'add', 'fe80::f816:3eff:fe79:188c/64', 'scope', 'link', 'dev', 'qg-e4629b84-e8'] execute_rootwrap_daemon /opt/stack/neutron/neutron/agent/linux/utils.py:100
2016-02-02 10:28:07.068 23328 ERROR neutron.agent.linux.utils [-] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-e4629b84-e8"

2016-02-02 10:28:07.069 23328 DEBUG oslo_concurrency.lockutils [-] Lock "l3-agent-pd" released by "neutron.agent.linux.pd.enable_subnet" :: held 0.055s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:282
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info [-] Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-e4629b84-e8"
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info Traceback (most recent call last):
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/common/utils.py", line 368, in call
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info return func(*args, **kwargs)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 736, in process
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info self._process_internal_ports(agent.pd)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 388, in _process_internal_ports
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info interface_name, p['mac_address'])
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info return f(*args, **kwargs)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 81, in enable_subnet
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info self._add_lla(router, pd_info.get_bind_lla_with_mask())
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 196, in _add_lla
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info 'link')
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/interface.py", line 194, in add_ipv6_addr
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info device.addr.add(str(net), scope)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 532, in add
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info self._as_root([net.version], tuple(args))
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 322, in _as_root
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info use_root_namespace=use_root_namespace)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 95, in _as_root
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info log_fail_as_error=self.log_fail_as_error)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 104, in _execute
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info log_fail_as_error=log_fail_as_error)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 140, in execute
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info raise RuntimeError(msg)
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-e4629b84-e8"
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info
2016-02-02 10:28:07.069 23328 ERROR neutron.agent.l3.router_info
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent [-] Failed to process compatible router 'f932415c-2cbd-4bcf-a54e-d7f7ee890908'
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 497, in _process_router_update
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent self._process_router_if_compatible(router)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 436, in _process_router_if_compatible
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent self._process_updated_router(router)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/agent.py", line 450, in _process_updated_router
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent ri.process(self)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/dvr_local_router.py", line 460, in process
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent super(DvrLocalRouter, self).process(agent)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/dvr_router_base.py", line 30, in process
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent super(DvrRouterBase, self).process(agent)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/common/utils.py", line 371, in call
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent self.logger(e)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/common/utils.py", line 368, in call
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent return func(*args, **kwargs)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 736, in process
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent self._process_internal_ports(agent.pd)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 388, in _process_internal_ports
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent interface_name, p['mac_address'])
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent return f(*args, **kwargs)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 81, in enable_subnet
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent self._add_lla(router, pd_info.get_bind_lla_with_mask())
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 196, in _add_lla
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent 'link')
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/interface.py", line 194, in add_ipv6_addr
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent device.addr.add(str(net), scope)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 532, in add
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent self._as_root([net.version], tuple(args))
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 322, in _as_root
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent use_root_namespace=use_root_namespace)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 95, in _as_root
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent log_fail_as_error=self.log_fail_as_error)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 104, in _execute
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent log_fail_as_error=log_fail_as_error)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 140, in execute
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent raise RuntimeError(msg)
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent RuntimeError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot find device "qg-e4629b84-e8"
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent
2016-02-02 10:28:07.075 23328 ERROR neutron.agent.l3.agent

That interface, qg-e4629b84-e8, lives in the SNAT namespace on a network node in dvr_snat mode.

Looking into it.

Brian Haley (brian-haley) on 2016-02-03
tags: added: l3-dvr-backlog
Carl Baldwin (carl-baldwin) on 2016-02-03
Changed in neutron:
status: New → Confirmed
Nate Johnston (nate-johnston) wrote :

Is this devstack-specific, or does it occur outside of devstack as well?

Changed in neutron:
importance: Undecided → Medium
John Davidge (john-davidge) wrote :

The note after the first paragraph of the Prefix Delegation section of [1] states that DVR is not supported, so failure is currently the expected behavior. I wonder if it's worth preventing the user from doing this with an error message instead of allowing the l3 agent to throw an exception.

[1] http://docs.openstack.org/liberty/networking-guide/adv_config_ipv6.html

Ritesh Anand (ritesh-anand) on 2016-02-03
Changed in neutron:
assignee: nobody → Ritesh Anand (ritesh-anand)
Brian Haley (brian-haley) wrote :

John - I completely missed that note, but we'll see what we can do to possibly fix it. If there's any info you can pass along I'd appreciate it.

Ritesh Anand (ritesh-anand) on 2016-02-05
Changed in neutron:
status: Confirmed → In Progress
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/277657

OpenStack Infra (hudson-openstack) on 2016-03-03
Changed in neutron:
assignee: Ritesh Anand (ritesh-anand) → Brian Haley (brian-haley)
OpenStack Infra (hudson-openstack) on 2016-03-04
Changed in neutron:
assignee: Brian Haley (brian-haley) → Ritesh Anand (ritesh-anand)
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/277657
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Armando Migliaccio (armando-migliaccio) on 2016-10-24
Changed in neutron:
assignee: Ritesh Anand (ritesh-anand) → nobody
status: In Progress → Confirmed
OpenStack Infra (hudson-openstack) on 2017-01-24
Changed in neutron:
assignee: nobody → Brian Haley (brian-haley)
status: Confirmed → In Progress
Armando Migliaccio (armando-migliaccio) on 2017-02-01
Changed in neutron:
milestone: none → ocata-rc1
tags: added: ocata-rc-potential
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/277657
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7b5f01c2e58cd21ab3d46676ee6a3288012fbb45
Submitter: Jenkins
Branch: master

commit 7b5f01c2e58cd21ab3d46676ee6a3288012fbb45
Author: Brian Haley <email address hidden>
Date: Tue Jan 24 13:34:58 2017 -0500

    Add IPv6 Prefix Delegation support for DVR

    Unlike Legacy routers, DVR Edge Routers have their gateway
    interfaces in the SNAT namespace as opposed to the router
    namespace. Added a new method to the router_info class,
    get_gw_ns_name(), so callers can determine which namespace
    the gateway device lives in. This can then be over-ridden
    in the DVR Edge router class.

    The Prefix Delegation code will also now listen on the
    update_router event from the l3-agent and reset the namespace
    name if it changes.

    Closes-Bug: #1541406
    Change-Id: If6ada5027d0483fac7fc3ff935fee1edfc6e2759

Changed in neutron:
status: In Progress → Fix Released
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 10.0.0.0rc1

This issue was fixed in the openstack/neutron 10.0.0.0rc1 release candidate.

sean redmond (sean-redmond1) wrote :

This impacts newton, is a backport sensible?

OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/463531

OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/newton)

Reviewed: https://review.openstack.org/463531
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a2f07de43f250b2114c9b4b074b61b4b4d8e450f
Submitter: Jenkins
Branch: stable/newton

commit a2f07de43f250b2114c9b4b074b61b4b4d8e450f
Author: Brian Haley <email address hidden>
Date: Tue Jan 24 13:34:58 2017 -0500

    Add IPv6 Prefix Delegation support for DVR

    Unlike Legacy routers, DVR Edge Routers have their gateway
    interfaces in the SNAT namespace as opposed to the router
    namespace. Added a new method to the router_info class,
    get_gw_ns_name(), so callers can determine which namespace
    the gateway device lives in. This can then be over-ridden
    in the DVR Edge router class.

    The Prefix Delegation code will also now listen on the
    update_router event from the l3-agent and reset the namespace
    name if it changes.

    Conflicts:
        neutron/tests/unit/agent/linux/test_pd.py

    Closes-Bug: #1541406
    Change-Id: If6ada5027d0483fac7fc3ff935fee1edfc6e2759
    (cherry picked from commit 7b5f01c2e58cd21ab3d46676ee6a3288012fbb45)

tags: added: in-stable-newton
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 9.4.0

This issue was fixed in the openstack/neutron 9.4.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.
Subscribing...

Other bug subscribers