Canonical Juju

"Prefer-IPv6" does not recognize IPv6 address

Bug #1767528 reported by Michael Quiniola
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Incomplete
Low
Unassigned
Charm Helpers
New
Undecided
Unassigned

Bug Description

I'm trying to build an openstack cloud based on IPv6 instead of IPv4 for better scalability.
Every charm being deployed is failing with the following error (directly from the logs).

Output of Unit Charm Log (In this particular case it's nova-compute):

2018-04-27 23:30:23 DEBUG juju-log Hardening function 'upgrade_charm'
2018-04-27 23:30:23 DEBUG juju-log Hardening function 'update_status'
2018-04-27 23:30:24 DEBUG juju-log No hardening applied to 'install'
2018-04-27 23:30:24 INFO juju-log DEPRECATION WARNING: Function configure_installation_source is being removed on/around 2017-07 : use charmhelpers.fetch.add_source() instead.
2018-04-27 23:30:24 DEBUG install Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
2018-04-27 23:30:25 DEBUG install Hit:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
2018-04-27 23:30:25 DEBUG install Hit:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
2018-04-27 23:30:25 DEBUG install Hit:4 http://archive.ubuntu.com/ubuntu bionic-security InRelease
2018-04-27 23:30:25 DEBUG install Fetched 242 kB in 1s (247 kB/s)
2018-04-27 23:30:26 DEBUG install Reading package lists...
2018-04-27 23:30:26 INFO juju-log Installing ['nova-compute', 'genisoimage', 'librbd1', 'python-six', 'python-psutil', 'nova-compute-kvm'] with options: ['--option=Dpkg::Options::=--force-confold']
2018-04-27 23:30:27 DEBUG install Reading package lists...
2018-04-27 23:30:27 DEBUG install Building dependency tree...
2018-04-27 23:30:27 DEBUG install Reading state information...
2018-04-27 23:30:27 DEBUG install genisoimage is already the newest version (9:1.1.11-3ubuntu2).
2018-04-27 23:30:27 DEBUG install librbd1 is already the newest version (12.2.4-0ubuntu1).
2018-04-27 23:30:27 DEBUG install nova-compute is already the newest version (2:17.0.1-0ubuntu1).
2018-04-27 23:30:27 DEBUG install nova-compute-kvm is already the newest version (2:17.0.1-0ubuntu1).
2018-04-27 23:30:27 DEBUG install python-psutil is already the newest version (5.4.2-1).
2018-04-27 23:30:27 DEBUG install python-six is already the newest version (1.11.0-2).
2018-04-27 23:30:27 DEBUG install 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
2018-04-27 23:30:30 DEBUG juju-log Generating template context for amqp
2018-04-27 23:30:31 DEBUG install ERROR no relation id specified
2018-04-27 23:30:31 DEBUG juju-log Generating template context for image-service.
2018-04-27 23:30:31 DEBUG juju-log 0 section(s) found
2018-04-27 23:30:32 INFO juju-log Address '10.0.0.51' is configured on iface 'br-eno1'
2018-04-27 23:30:32 DEBUG install Traceback (most recent call last):
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/install.real", line 534, in <module>
2018-04-27 23:30:32 DEBUG install main()
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/install.real", line 530, in main
2018-04-27 23:30:32 DEBUG install assess_status(CONFIGS)
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/nova_compute_utils.py", line 735, in assess_status
2018-04-27 23:30:32 DEBUG install assess_status_func(configs)()
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1405, in _assess_status_func
2018-04-27 23:30:32 DEBUG install state, message = _determine_os_workload_status(*args, **kwargs)
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 808, in _determine_os_workload_status
2018-04-27 23:30:32 DEBUG install configs, required_interfaces)
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 862, in _ows_check_generic_interfaces
2018-04-27 23:30:32 DEBUG install required_interfaces)
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1143, in incomplete_relation_data
2018-04-27 23:30:32 DEBUG install complete_ctxts = configs.complete_contexts()
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 351, in complete_contexts
2018-04-27 23:30:32 DEBUG install for i in six.itervalues(self.templates)]
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 351, in <listcomp>
2018-04-27 23:30:32 DEBUG install for i in six.itervalues(self.templates)]
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 127, in complete_contexts
2018-04-27 23:30:32 DEBUG install self.context()
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 112, in context
2018-04-27 23:30:32 DEBUG install _ctxt = context()
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/nova_compute_context.py", line 581, in __call__
2018-04-27 23:30:32 DEBUG install ctxt['console_listen_addr'] = resolve_address(endpoint_type=INTERNAL)
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/openstack/ip.py", line 167, in resolve_address
2018-04-27 23:30:32 DEBUG install fallback_addr = get_ipv6_addr(exc_list=vips)[0]
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/network/ip.py", line 353, in iface_sniffer
2018-04-27 23:30:32 DEBUG install return f(*args, **kwargs)
2018-04-27 23:30:32 DEBUG install File "/var/lib/juju/agents/unit-nova-compute-1/charm/hooks/charmhelpers/contrib/network/ip.py", line 421, in get_ipv6_addr
2018-04-27 23:30:32 DEBUG install "non-temporary ipv6 address." % iface)
2018-04-27 23:30:32 DEBUG install Exception: Interface 'br-eno1' does not have a scope global non-temporary ipv6 address.
2018-04-27 23:30:32 ERROR juju.worker.uniter.operation runhook.go:113 hook "install" failed: exit status 1

My IPv6 is a static setting using ULA (equivalent to a 192.168.0.0) address:

Output of ifconfig:

br-eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 10.0.0.51 netmask 255.255.255.0 broadcast 10.0.0.255
        inet6 fd00::1:0:4 prefixlen 64 scopeid 0x0<global>
        inet6 fe80::7cea:d7ff:fe95:31c prefixlen 64 scopeid 0x20<link>
        ether 7e:ea:d7:95:03:1c txqueuelen 1000 (Ethernet)
        RX packets 5578 bytes 3104832 (3.1 MB)
        RX errors 0 dropped 1 overruns 0 frame 0
        TX packets 3863 bytes 696147 (696.1 KB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Output of /etc/netplan:

network:
  version: 2
  ethernets:
    eno1:
      match:
        macaddress: 00:25:90:4c:79:58
      set-name: eno1
      mtu: 1500
    eno2:
      match:
        macaddress: 00:25:90:4c:79:59
      set-name: eno2
      mtu: 1500
  bridges:
    br-eno1:
      interfaces: [eno1]
      addresses:
      - 10.0.0.51/24
      - fd00::1:0:4/64
      gateway4: 10.0.0.1
      gateway6: fd00::1
      nameservers:
        addresses: [10.0.0.2, 'fd00::2']
      mtu: 1500

I'm well aware that Bionic LTS was just released, but this is as far as I have gotten on xenial LTS as well. Same exact issues.

I have yet to tackle the configuration issue of LXCs, this is just the bare metal level to start.

Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 1767528] [NEW] "Prefer-IPv6" does not recognize IPv6 address
Download full text (8.8 KiB)

The device in question indeed does not have a 'scope global' address:
https://en.wikipedia.org/wiki/Unique_local_address

says that fd00::/8 is a 'unique local address'.
I'm not sure why charmhelpers isn't happy to use them.

Note, though that the description there says you probably shouldn't use
fd00::/8 directly, but should use a randomly-generated prefix.

Generally it seems like it is a charmhelpers issue, rather than a Juju
issue.

On Sat, Apr 28, 2018 at 3:39 AM, Michael Quiniola <
<email address hidden>> wrote:

> Public bug reported:
>
> I'm trying to build an openstack cloud based on IPv6 instead of IPv4 for
> better scalability.
> Every charm being deployed is failing with the following error (directly
> from the logs).
>
>
> Output of Unit Charm Log (In this particular case it's nova-compute):
>
> 2018-04-27 23:30:23 DEBUG juju-log Hardening function 'upgrade_charm'
> 2018-04-27 23:30:23 DEBUG juju-log Hardening function 'update_status'
> 2018-04-27 23:30:24 DEBUG juju-log No hardening applied to 'install'
> 2018-04-27 23:30:24 INFO juju-log DEPRECATION WARNING: Function
> configure_installation_source is being removed on/around 2017-07 : use
> charmhelpers.fetch.add_source() instead.
> 2018-04-27 23:30:24 DEBUG install Get:1 http://archive.ubuntu.com/ubuntu
> bionic InRelease [242 kB]
> 2018-04-27 23:30:25 DEBUG install Hit:2 http://archive.ubuntu.com/ubuntu
> bionic-updates InRelease
> 2018-04-27 23:30:25 DEBUG install Hit:3 http://archive.ubuntu.com/ubuntu
> bionic-backports InRelease
> 2018-04-27 23:30:25 DEBUG install Hit:4 http://archive.ubuntu.com/ubuntu
> bionic-security InRelease
> 2018-04-27 23:30:25 DEBUG install Fetched 242 kB in 1s (247 kB/s)
> 2018-04-27 23:30:26 DEBUG install Reading package lists...
> 2018-04-27 23:30:26 INFO juju-log Installing ['nova-compute',
> 'genisoimage', 'librbd1', 'python-six', 'python-psutil',
> 'nova-compute-kvm'] with options: ['--option=Dpkg::Options::=--
> force-confold']
> 2018-04-27 23:30:27 DEBUG install Reading package lists...
> 2018-04-27 23:30:27 DEBUG install Building dependency tree...
> 2018-04-27 23:30:27 DEBUG install Reading state information...
> 2018-04-27 23:30:27 DEBUG install genisoimage is already the newest
> version (9:1.1.11-3ubuntu2).
> 2018-04-27 23:30:27 DEBUG install librbd1 is already the newest version
> (12.2.4-0ubuntu1).
> 2018-04-27 23:30:27 DEBUG install nova-compute is already the newest
> version (2:17.0.1-0ubuntu1).
> 2018-04-27 23:30:27 DEBUG install nova-compute-kvm is already the newest
> version (2:17.0.1-0ubuntu1).
> 2018-04-27 23:30:27 DEBUG install python-psutil is already the newest
> version (5.4.2-1).
> 2018-04-27 23:30:27 DEBUG install python-six is already the newest version
> (1.11.0-2).
> 2018-04-27 23:30:27 DEBUG install 0 upgraded, 0 newly installed, 0 to
> remove and 0 not upgraded.
> 2018-04-27 23:30:30 DEBUG juju-log Generating template context for amqp
> 2018-04-27 23:30:31 DEBUG install ERROR no relation id specified
> 2018-04-27 23:30:31 DEBUG juju-log Generating template context for
> image-service.
> 2018-04-27 23:30:31 DEBUG juju-log 0 section(s) found
> 2018-04-27 23:30:32 INFO juju-log Address '10.0.0.51' is configured...

Read more...

Changed in juju:
importance: Undecided → High
status: New → Incomplete
Revision history for this message
John A Meinel (jameinel) wrote :

Can you also give the output of:

 juju status --format=yaml
and
 juju run --unit nova-compute/? network-get

Revision history for this message
John A Meinel (jameinel) wrote :

It does seem like you are getting both a local address (fd00::1:0:4) and a possibly valid address (fe80::7cea:d7ff:fe95:31c).
Presumably the latter is via auto-discovery?
Or is it being defined in MAAS?

Can you give a little bit more information about your configuration? (have you tried deploying the applications by themselves rather than all of openstack to make it a bit easier to reproduce?)
We'd also need how you're configuring the nodes in MAAS.

Revision history for this message
Michael Quiniola (qthepirate) wrote :

This bug still exists.

Originally you were correct, I was using a ULA address.
But now, I am using proper public global IPv6 addresses. I wont post them here, but during install of, for example, the nova-compute charm, I still receive the error.

What it LOOKS like, is that after juju discovers an IPv4 address (I have dual stack), it errors. What this tells me is that it may be looking for an IPv6 address only and if an IPv4 address is found it assumes there is no IPv6 and it immediately fails.

Revision history for this message
Canonical Juju QA Bot (juju-qa-bot) wrote :

This bug has not been updated in 2 years, so we're marking it Low importance. If you believe this is incorrect, please update the importance.

Changed in juju:
importance: High → Low
tags: added: expirebugs-bot
Revision history for this message
Michael Quiniola (qthepirate) wrote :

This eventually came down to IPv6 support for LXCs via Juju. Whether it's been resolved or not I am not sure.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.