On host systems with IPv6 enabled, install fails at overcloud_prep_network step due to ipv6 dns server being used for ipv4 neutron network

[David Peacock]

CentOS 7 host system, with IPv6 enabled
OpenStack environment: Latest upstream via latest tripleo-quickstart devmode.sh

DNS resolver set thus:

[dpeacock@gandalf ~]$ cat /etc/resolv.conf
# Generated by NetworkManager
search davidjpeacock.ca
nameserver 192.168.1.1
nameserver 2001:4860:4860:8888::
nameserver 2001:4860:4860:8844::

tripleo-quickstart kicked off against this host via:

`$ ./devmode.sh --no-gate gandalf`

Ansible failure information:

```
TASK [overcloud-prep-network : Create overcloud-prep-network script] ***********************************
task path: /home/dpeacock/.quickstart/usr/local/share/ansible/roles/overcloud-prep-network/tasks/create-scripts.yml:3
Monday 04 December 2017 16:28:43 -0500 (0:01:09.077) 1:06:14.707 *******
changed: [undercloud] => {"changed": true, "checksum": "ddd8237240fd5f099ca0ed618f8a3af9fda6e94a", "dest": "/home/stack/overcloud-prep-network.sh", "failed": false, "gid": 1001, "group": "stack", "md5sum": "95744bfdb86952408b7ead37d3a07ce1", "mode": "0755", "owner": "stack", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 1450, "src": "/home/stack/.ansible/tmp/ansible-tmp-1512422924.04-183330380951554/source", "state": "file", "uid": 1000}

TASK [overcloud-prep-network : Prepare the network-isolation required networks on the undercloud] ******
task path: /home/dpeacock/.quickstart/usr/local/share/ansible/roles/overcloud-prep-network/tasks/overcloud-prep-network.yml:1
Monday 04 December 2017 16:29:22 -0500 (0:00:38.915) 1:06:53.622 *******
fatal: [undercloud]: FAILED! => {"changed": true, "cmd": "set -o pipefail && /home/stack/overcloud-prep-network.sh 2>&1 | awk '{ print strftime(\"%Y-%m-%d %H:%M:%S |\"), $0; fflush(); }' > /home/stack/overcloud_prep_network.log", "delta": "0:00:10.712405", "end": "2017-12-04 21:29:57.486725", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2017-12-04 21:29:46.774320", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

PLAY RECAP *********************************************************************************************
gandalf : ok=156 changed=75 unreachable=0 failed=0
localhost : ok=17 changed=8 unreachable=0 failed=1
undercloud : ok=49 changed=34 unreachable=0 failed=1

```

Log of failure from undercloud virt:

```
[stack@undercloud ~]$ tail -20 overcloud_prep_network.log
2017-12-04 21:29:46 | DEVICETYPE=ovs
2017-12-04 21:29:46 | TYPE=OVSIntPort
2017-12-04 21:29:46 | BOOTPROTO=static
2017-12-04 21:29:46 | IPADDR=10.0.0.1
2017-12-04 21:29:46 | NETMASK=255.255.255.0
2017-12-04 21:29:46 | OVS_BRIDGE=br-ctlplane
2017-12-04 21:29:46 | OVS_OPTIONS="tag=10"
2017-12-04 21:29:46 | EOF'
2017-12-04 21:29:46 | + sudo ifup ifcfg-vlan10
2017-12-04 21:29:51 | ++ neutron net-list
2017-12-04 21:29:51 | ++ awk '/ctlplane/{print $(NF-2) }'
2017-12-04 21:29:51 | neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
2017-12-04 21:29:54 | ++ cat /etc/resolv.conf
2017-12-04 21:29:54 | ++ grep nameserver
2017-12-04 21:29:54 | ++ awk '{ print $2 }'
2017-12-04 21:29:54 | ++ sed ':a;N;$!ba;s/\n/ --dns-nameserver /g'
2017-12-04 21:29:54 | + neutron subnet-update f115ae1b-7c8a-43a3-b8ab-aee174d459f4 --dns-nameserver 192.168.1.1 --dns-nameserver 2001:4860:4860:8888::
2017-12-04 21:29:55 | neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
2017-12-04 21:29:57 | Invalid input for operation: dns_nameserver '2001:4860:4860:8888::' does not match the ip_version '4'.
2017-12-04 21:29:57 | Neutron server returns request_ids: ['req-b7b17c8a-9776-4eb9-ae98-d22a3395dd28']
```

Expected result:

Correct setup for DNS, and ultimate successful deployment.

Actual result:

Failure due to confusion of dns_nameserver not being an IPv4 address.

[wes hayutin (weshayutin)]

Are you attempting to deploy the overcloud using ipv6?
If you are passing the following config to devmode [1]

[1] https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset024.yml

[Emilien Macchi (emilienm)]

Wes, I don't think he's trying to deploy TripleO with an ipv6 isolated network, rather on a host that has ipv6 enabled, which is a different case.
I'm re-opening the bug report.

[David Peacock (davidjpeacock)]

Hi Wes,

My apologies for not being more clear; I'm actually attempting to deploy a regular tripleo-quickstart environment via the `devstack.sh` script, basically default vanilla style.

The IPv6 comes into play in that the host machine I am running this on has a dual-stack IPv4 and IPv6 setup.

```
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 44:39:c4:51:63:48 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.210/24 brd 192.168.1.255 scope global eno1
       valid_lft forever preferred_lft forever
    inet6 2001:1970:4ba8:6500:5bdc:1c5:9b7a:8c3a/64 scope global noprefixroute dynamic
       valid_lft 65395sec preferred_lft 22195sec
    inet6 fe80::3d6c:e07f:8e24:866c/64 scope link
       valid_lft forever preferred_lft forever
```

Please let me know if there are any details, logs, commands, etc, you need me to provide.

Thank you!
David :-)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/534326

[Martin André (mandre)]

I faced the same issue today (which is odd because I had IPv6 enabled since always). https://review.openstack.org/#/c/534326/ should fix it.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/534326
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=770ef97a9d36ed8dcc8da227af46fbc255b9b88b
Submitter: Zuul
Branch: master

commit 770ef97a9d36ed8dcc8da227af46fbc255b9b88b
Author: Martin André <email address hidden>
Date: Tue Jan 16 15:20:57 2018 +0100

    Restrict resolvers to IPv4 addresses

    Previously, the overcloud-prep-network.sh script would parse
    /etc/resolv.conf and add all resolvers found in the file to the ctlplane
    neutron subnet unless the overcloud_dns_servers variable is set.

    In the case the host machine has IPv6 enabled, this causes the neutron
    subnet-update command to fail on IPv6 addresses with:

      Invalid input for operation: dns_nameserver '2a01:e00::1' does not
      match the ip_version '4'.

    This commit filters out IPv6 addresses.

    Change-Id: I4f287015b6161b615eb95312b528d6a392a54d90
    Closes-Bug: #1736264

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-quickstart-extras 2.1.1 release.