Cannot enable DVR and IPv6 simultaneously

[Brian Haley]

While testing out the devstack change to support IPv6, https://review.openstack.org/#/c/87987/ I tripped-over a DVR error since I have it enabled by default in local.conf.

I have these two things enabled in local.conf:

Q_DVR_MODE=dvr_snat
IP_VERSION=4+6

After locally fixing lib/neutron to teach it about the DVR snat- namespace (another bug to be filed for that), stack.sh was able to complete, but the l3-agent wasn't very happy:

Stderr: '' execute /opt/stack/neutron/neutron/agent/linux/utils.py:81
2014-09-30 12:53:47.511 21778 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-95b180a4-9623-4ef9-adda-772ca0838253', 'ip', 'rule', 'add', 'from', 'fd00::1/64', 'lookup', '336294682933583715844663186250927177729', 'priority', '336294682933583715844663186250927177729'] create_process /opt/stack/neutron/neutron/agent/linux/utils.py:46
2014-09-30 12:53:47.641 21778 ERROR neutron.agent.linux.utils [-]
Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-95b180a4-9623-4ef9-adda-772ca0838253', 'arping', '-A', '-I', 'qr-3d0eda6e-54', '-c', '3', 'fd00::1']
Exit code: 2
Stdout: ''
Stderr: 'arping: unknown host fd00::1\n'
2014-09-30 12:53:47.643 21778 ERROR neutron.agent.l3_agent [-] Failed sending gratuitous ARP:
Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-95b180a4-9623-4ef9-adda-772ca0838253', 'arping', '-A', '-I', 'qr-3d0eda6e-54', '-c', '3', 'fd00::1']
Exit code: 2
Stdout: ''
Stderr: 'arping: unknown host fd00::1\n'
2014-09-30 12:53:48.682 21778 ERROR neutron.agent.linux.utils [-]
Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-95b180a4-9623-4ef9-adda-772ca0838253', 'ip', 'rule', 'add', 'from', 'fd00::1/64', 'lookup', '336294682933583715844663186250927177729', 'priority', '336294682933583715844663186250927177729']
Exit code: 255
Stdout: ''
Stderr: 'Error: argument "336294682933583715844663186250927177729" is wrong: preference value is invalid\n\n'
2014-09-30 12:53:48.683 21778 ERROR neutron.agent.l3_agent [-] DVR: error adding redirection logic
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent Traceback (most recent call last):
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent File "/opt/stack/neutron/neutron/agent/l3_agent.py", line 1443, in _snat_redirect_add
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent ns_ipr.add_rule_from(sn_port['ip_cidr'], snat_idx, snat_idx)
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 202, in add_rule_from
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent ip = self._as_root('', 'rule', tuple(args))
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 74, in _as_root
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent log_fail_as_error=self.log_fail_as_error)
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 86, in _execute
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent log_fail_as_error=log_fail_as_error)
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 84, in execute
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent raise RuntimeError(m)
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent RuntimeError:
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-95b180a4-9623-4ef9-adda-772ca0838253', 'ip', 'rule', 'add', 'from', 'fd00::1/64', 'lookup', '336294682933583715844663186250927177729', 'priority', '336294682933583715844663186250927177729']
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent Exit code: 255
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent Stdout: ''
2014-09-30 12:53:48.683 21778 TRACE neutron.agent.l3_agent Stderr: 'Error: argument "336294682933583715844663186250927177729" is wrong: preference value is invalid\n\n'

Ignore the ARP failures, there's already an upstream patch proposed for that.

The fix for now might just be to ignore IPv6 addresses in the SNAT code, we can look at optimizations later, but need to get this working so we can enable both at the same time.

There are other errors that this then triggers, so devstack isn't very usable until you turn one off.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/134676

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/136947

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/138654

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/141297

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/141297
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=deffbbfdfef57f740006215674a7aa54a5a1f5cc
Submitter: Jenkins
Branch: master

commit deffbbfdfef57f740006215674a7aa54a5a1f5cc
Author: Xu Han Peng <email address hidden>
Date: Wed Dec 10 14:07:42 2014 +0800

    Fix IPv6 RA security group rule for DVR

    Current IPv6 RA security group rule doesn't work for
    DVR because the code only allows RA from device_owner
    is network:router_interface. When DVR is enabled, the
    router interface is network:router_interface_distributed.

    This fix fixes the RA rule to allow RA from DVR router
    interface, so router advertisement from DVR RADVD can
    pass to VM.

    Co-Authored-By: Baodong (Robert) Li <email address hidden>

    Change-Id: Idd1324c653dcb15f5dacf2d897a7048bca22fc38
    Partial-Bug: 1376325

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/138654
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=727417e71ed155c7d356b27518896026338f79c3
Submitter: Jenkins
Branch: master

commit 727417e71ed155c7d356b27518896026338f79c3
Author: Xu Han Peng <email address hidden>
Date: Wed Dec 3 14:58:34 2014 +0800

    Fix DVR flow problems for IPv6 subnet

    This code fixes DVR flow problems by changing proto='ip' to
    proto='ipv6' and changing nw_dst to ipv6_dst.

    When DVR is enabled, RADVD is spawned by l3 agent on each compute
    node. This code also prevent IPv6 Router Advertisement from
    sending to other compute nodes.

    Change-Id: Id94acd85ea124eff6cfdfbfc546f5dd4ca81ef43
    Closes-Bug: 1398244
    Closes-Bug: 1398627
    Partial-Bug: 1376325

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/134676
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7eb23f662cd62f29700ae51ee16dc535d251fc27
Submitter: Jenkins
Branch: master

commit 7eb23f662cd62f29700ae51ee16dc535d251fc27
Author: rajeev <email address hidden>
Date: Wed Nov 12 11:25:55 2014 -0500

    Add index generation for IPv6 rules for DVR

    For IPv6 support with DVR the index used for rule priority and
    route table needs to be generated such that the index is 32 bits
    or less but greater than the system generated rule entries.

    For IPv4 the numeric value of the network is used as the index.
    For IPv6 the 30 bit xor-folded crc32 of the cidr is used.
    Values smaller than system generated entries are extended into
    the range freed up because of xor-folding.
    For code modularity, index generation is wrapped into a helper
    method.

    Partial-bug: #1376325

    Change-Id: I4bcde80257ef5fe7c744086019841cdef5e3c60c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/136947
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4d774ef13cc4292f4fd95082ec1ceb0db24b39c7
Submitter: Jenkins
Branch: master

commit 4d774ef13cc4292f4fd95082ec1ceb0db24b39c7
Author: Brian Haley <email address hidden>
Date: Mon Nov 24 21:33:20 2014 -0500

    Add address family to 'ip rule' calls

    Without an address family, 'ip rule' won't work with IPv6
    arguments because it assumes IPv4. This causes the l3-agent
    to throw an error when adding a rule in DVR mode.

    Also changed these functions to be more symmetrical and take the
    same arguments, which required a little tweaking, but it looks
    much cleaner now.

    Change-Id: I85718d8d6ffcf3dec2a6b92641a731af813114aa
    Closes-bug: #1376325

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/154576

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/154576
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=edf0aab9169b435eb0c458b5086afd0b074b25fa
Submitter: Jenkins
Branch: stable/juno

commit edf0aab9169b435eb0c458b5086afd0b074b25fa
Author: Xu Han Peng <email address hidden>
Date: Wed Dec 10 14:07:42 2014 +0800

    Fix IPv6 RA security group rule for DVR

    Current IPv6 RA security group rule doesn't work for
    DVR because the code only allows RA from device_owner
    is network:router_interface. When DVR is enabled, the
    router interface is network:router_interface_distributed.

    This fix fixes the RA rule to allow RA from DVR router
    interface, so router advertisement from DVR RADVD can
    pass to VM.

    Co-Authored-By: Baodong (Robert) Li <email address hidden>

    Change-Id: Idd1324c653dcb15f5dacf2d897a7048bca22fc38
    Partial-Bug: 1376325
    Cherry-Picked-From: deffbbfdfef57f740006215674a7aa54a5a1f5cc