When creating a server instance with an IPv4 and an IPv6 addresses, the IPv6 is not assigned

[Federico Ressi]

This is expected behaviour:

Given:
  G1) A tenant network with two subnets (one for IPV4 and one for IPv6)
  G2) A port attached to the network with two fixed IPs (one for IPv4 and one for IPv6)
  G3) A server VM instance created with the port with its two fixed IPs

When:
  W1) Server instance is booted

Then:
  T1) Server instance receives from cloud init the IPv4 address correctly on the first interface and set it up
  T2) Server instance receives from cloud init the IPv6 address correctly on the first interface and set it up

The observed behavior differs in T2:
  T2) Server instance receives from cloud init the wrong IPv6 on the first interface and set it up the wrong IP

These are the IPs of the server when it is created:

  "addresses": {
    "tempest-loginable-619471459": [
        {"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:81:72:35",
         "version": 4, "addr": "10.1.0.35", "OS-EXT-IPS:type": "fixed"},
        {"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:81:72:35",
         "version": 6, "addr": "2003:0:0:2::2", "OS-EXT-IPS:type": "fixed"}
    ]
  }

These are the actual IPs assigned to the VM:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:81:72:35 brd ff:ff:ff:ff:ff:ff
    inet 10.1.0.35/28 brd 10.1.0.47 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe81:7235/64 scope link
       valid_lft forever preferred_lft forever

These traces has been extracted from the instance log:

[ 331.846900] cloud-init[1014]: Cloud-init v. 18.2 running 'init' at Tue, 25 Sep 2018 19:13:18 +0000. Up 262.77 seconds.
    [ 331.986855] cloud-init[1014]: ci-info: +++++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++++++
    [ 332.026098] cloud-init[1014]: ci-info: +--------+------+------------------------------+-----------------+-------+-------------------+
    [ 332.052076] cloud-init[1014]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
    [ 332.117348] cloud-init[1014]: ci-info: +--------+------+------------------------------+-----------------+-------+-------------------+
    [ 332.137446] cloud-init[1014]: ci-info: | ens3 | True | 10.1.0.35 | 255.255.255.240 | . | fa:16:3e:81:72:35 |
    [ 332.166719] cloud-init[1014]: ci-info: | ens3 | True | fe80::f816:3eff:fe81:7235/64 | . | link | fa:16:3e:81:72:35 |
    [ 332.197524] cloud-init[1014]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | . |
    [ 332.223152] cloud-init[1014]: ci-info: | lo | True | ::1/128 | . | host | . |
    [ 332.258243] cloud-init[1014]: ci-info: +--------+------+------------------------------+-----------------+-------+-------------------+
    [ 332.282758] cloud-init[1014]: ci-info: +++++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++++
    [ 332.318255] cloud-init[1014]: ci-info: +-------+-----------------+-----------+-----------------+-----------+-------+
    [ 332.343051] cloud-init[1014]: ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
    [ 332.369270] cloud-init[1014]: ci-info: +-------+-----------------+-----------+-----------------+-----------+-------+
    [ 332.394200] cloud-init[1014]: ci-info: | 0 | 0.0.0.0 | 10.1.0.33 | 0.0.0.0 | ens3 | UG |
    [ 332.418340] cloud-init[1014]: ci-info: | 1 | 10.1.0.32 | 0.0.0.0 | 255.255.255.240 | ens3 | U |
    [ 332.442581] cloud-init[1014]: ci-info: | 2 | 169.254.169.254 | 10.1.0.33 | 255.255.255.255 | ens3 | UGH |
    [ 332.463209] cloud-init[1014]: ci-info: +-------+-----------------+-----------+-----------------+-----------+-------+

This bug has been see when writing this test case for neutron-tempest-plugin:
  https://review.openstack.org/#/c/586040/21
  https://review.openstack.org/#/c/586040/21/neutron_tempest_plugin/scenario/test_loginable.py

The logs of the problem can be found here:
  http://logs.openstack.org/40/586040/21/check/neutron-tempest-plugin-scenario-linuxbridge/28228b0/
  http://logs.openstack.org/40/586040/21/check/neutron-tempest-plugin-scenario-linuxbridge/28228b0/testr_results.html.gz

[Brian Haley (brian-haley)]

So it looks like it's getting the IPv6 link-local address, but not the global. Does 'openstack server list' show the IPv6 address assigned and/or is there a neutron port for it?

This bug looks related, but it should have been fixed in the version you're using - https://bugs.launchpad.net/cirros/+bug/1459042

[Federico Ressi (fressi-redhat)]

According to "Using OpenStack Networking with IPv6"[1] document what I was doing wrong is not passing two parameters when creating IPV6 subnet:

- ipv6_ra_mode
- ipv6_address_mode

Setting both to one of below values it looks like successfully assigning the IPV6 address:
 - slaac
 - dhcpv6-stateless

Setting both of them to 'dhcpv6-stateful' instead doesn't assign the right IPV6 address to the network interface.

[1] https://docs.openstack.org/liberty/networking-guide/adv-config-ipv6.html

As this is a documented behavior then this is not a bug except for the case of passing to both parameters the value 'dhcpvs-stateful', in which case the expected behavior is:

"Guest instance obtains IPv6 address from dnsmasq using DHCPv6 stateful and optional info from dnsmasq using DHCPv6."

I don't have elements in this moment to understand if above sentence actually matches with my test results.

[Brian Haley (brian-haley)]

When both are set to 'dhcpv6-stateful', the instance must run a dhcpv6 client in order to get it's global IPv6 address. In both of the other two modes it will use SLAAC to auto-configure it's global address based on the information in the router advertisement, which most images (even cirros) will do correctly.

My guess is that maybe you set ipv6-address-mode but not ipv6-ra-mode, so even though neutron knew what the address should be, radvd was never started so SLAAC never occurred, leading to this failure.

[Slawek Kaplonski (slaweq)]

Checking logs pointed by Federico in bug report I found something like:

"dnsmasq-dhcp[1891]: duplicate IP address 2003:0:0:2::2 (server-name) in dhcp-config directive"

http://logs.openstack.org/40/586040/21/check/neutron-tempest-plugin-scenario-linuxbridge/28228b0/controller/logs/screen-q-dhcp.txt.gz#_Sep_25_19_08_46_602298

But I'm not sure if that happens also in other cases and if that might be reason of this issue or not.

[Federico Ressi (fressi-redhat)]

I was finally able to ping between two VMs using IPv6 subnet by doing below:

1) Add below parameters when creating IPv6 subnet:

  {
    'enable_dhcp': true,
    'ipv6_ra_mode':'slaac',
    'ipv6_address_mode':'slaac',
    # ...
  }

2) Add below rule to security group assigned to VMs ports:

  {
    'protocol': 'icmp',
    'direction': 'ingress',
    'remote_ip_prefix': '::/0',
    'ethertype': 'IPv6'
  }

Cheers for your help.

[Slawek Kaplonski (slaweq)]

I checked today this issue with dhcpv6-stateful locally. Everything works fine there, only problem is that in case of using DHCPv6 to get IPv6 address, You need to run dhclient for IPv6. On Ubuntu 16.04 it's something like:

dhclient -6 ens4

and this isn't run automatically when "ip link set up ens4" is done.
So I will close this as invalid because all works fine here IMO.