OSError: [Errno 97] Address family not supported by protocol

[Bruno Carvalho]

After disabling IPV6 support in my Ubuntu 16.4 kernel, MAAS 2.1.3 does not work.

Feb 9 18:17:30 trm-l-srv-149 systemd[1]: [e-mail]: Service hold-off time over, scheduling restart.
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: [e-mail]: Service hold-off time over, scheduling restart.
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Stopped MAAS Region Controller (Worker 1).
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Started MAAS Region Controller (Worker 1).
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Stopped MAAS Region Controller (Worker 3).
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Started MAAS Region Controller (Worker 3).
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: maas-rackd.service: Service hold-off time over, scheduling restart.
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Stopped MAAS Rack Controller.
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Starting MAAS Rack Controller...
Feb 9 18:17:30 trm-l-srv-149 systemd[1]: Started MAAS Rack Controller.
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: Traceback (most recent call last):
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/bin/twistd3", line 18, in <module>
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: run()
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/twisted/scripts/twistd.py", line 29, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: app.run(runApp, ServerOptions)
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 617, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: runApp(config)
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/twisted/scripts/twistd.py", line 25, in runApp
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: _SomeApplicationRunner(config).run()
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 348, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: self.application = self.createOrGetApplication()
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 408, in createOrGetApplication
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: ser = plg.makeService(self.config.subOptions)
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/provisioningserver/plugin.py", line 200, in makeService
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: for service in self._makeServices(tftp_root, tftp_port):
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/provisioningserver/plugin.py", line 173, in _makeServices
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: yield self._makeImageService(tftp_root)
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3/dist-packages/provisioningserver/plugin.py", line 66, in _makeImageService
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: File "/usr/lib/python3.5/socket.py", line 134, in __init__
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: _socket.socket.__init__(self, family, type, proto, fileno)
Feb 9 18:17:31 trm-l-srv-149 sh[5169]: OSError: [Errno 97] Address family not supported by protocol
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: Traceback (most recent call last):
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 306, in populateService
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: service = self.services.getServiceNamed(name)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/twisted/application/service.py", line 295, in getServiceNamed
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: return self.namedServices[name]
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: KeyError: 'web'
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: During handling of the above exception, another exception occurred:
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: Traceback (most recent call last):
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/bin/twistd3", line 18, in <module>
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: run()
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/twisted/scripts/twistd.py", line 29, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: app.run(runApp, ServerOptions)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 617, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: runApp(config)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/twisted/scripts/twistd.py", line 25, in runApp
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: _SomeApplicationRunner(config).run()
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 348, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: self.application = self.createOrGetApplication()
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 408, in createOrGetApplication
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: ser = plg.makeService(self.config.subOptions)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/maasserver/plugin.py", line 118, in makeService
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: eventloop.loop.populate()
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 153, in wrapper
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: return func(*args, **kwargs)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 328, in populate
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: self.populateService(name)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 153, in wrapper
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: return func(*args, **kwargs)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 314, in populateService
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: service = factoryInfo["factory"](*dependencies)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 164, in make_WebApplicationService
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: File "/usr/lib/python3.5/socket.py", line 134, in __init__
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: _socket.socket.__init__(self, family, type, proto, fileno)
Feb 9 18:17:31 trm-l-srv-149 sh[5149]: OSError: [Errno 97] Address family not supported by protocol
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: Traceback (most recent call last):
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 306, in populateService
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: service = self.services.getServiceNamed(name)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/twisted/application/service.py", line 295, in getServiceNamed
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: return self.namedServices[name]
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: KeyError: 'web'
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: During handling of the above exception, another exception occurred:
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: Traceback (most recent call last):
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/bin/twistd3", line 18, in <module>
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: run()
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/twisted/scripts/twistd.py", line 29, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: app.run(runApp, ServerOptions)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 617, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: runApp(config)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/twisted/scripts/twistd.py", line 25, in runApp
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: _SomeApplicationRunner(config).run()
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 348, in run
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: self.application = self.createOrGetApplication()
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 408, in createOrGetApplication
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: ser = plg.makeService(self.config.subOptions)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/maasserver/plugin.py", line 118, in makeService
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: eventloop.loop.populate()
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 153, in wrapper
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: return func(*args, **kwargs)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 324, in populate
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: self.populateService(name)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/provisioningserver/utils/twisted.py", line 153, in wrapper
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: return func(*args, **kwargs)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 314, in populateService
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: service = factoryInfo["factory"](*dependencies)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3/dist-packages/maasserver/eventloop.py", line 164, in make_WebApplicationService
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: File "/usr/lib/python3.5/socket.py", line 134, in __init__
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: _socket.socket.__init__(self, family, type, proto, fileno)
Feb 9 18:17:31 trm-l-srv-149 sh[5148]: OSError: [Errno 97] Address family not supported by protocol

[Andres Rodriguez (andreserl)]

Hi Bruno,

Could you please provide what you have in :

/etc/maas/regiond.conf
/etc/maas/rackd.conf
/var/log/maas/*.log
/etc/hosts

That said, can you provide exact steps to reproduce? (I.e. How did you disable IPv6 and whether you restarted Maas or not)

Other than that, have you tried restarting Maas services?

[Mike Pontillo (mpontillo)]

In order to support IPv6-only and IPv4/IPv6 dual-stack environments, MAAS now makes use of IPv4-mapped IPv6 addresses for its basic operation. Normally this ensures compatibility with all combinations of address families.

Being that IPv6 has been in development since 1995 and has been in the Linux kernel since 1996, I don't think MAAS should need to assume it doesn't exist.

Can you use some other mechanism to disable IPv6 in your environment, such as disabling the acceptance of IPv6 router advertisements?

[Bruno Carvalho (brunowcs)]

Mike,

I am disable ipv6 via boot kernel, but you speek correct, the MAAS need the ipv6

Feb 9 19:58:25 trm-l-srv-90 sh[14673]: 2017-02-09 19:58:25 twisted.internet.protocol.Factory: [info] RegionServer connection established (HOST:IPv6Address(TCP, '::ffff:172.16.179.12', 5252) PEER:IPv6Address(TCP, '::ffff:172.16.179.14', 37688))

IPV6 ENABLED = MAAS OK

Tks

[Mike Pontillo (mpontillo)]

Bruno, how exactly are you disabling IPv6? (And why are you doing it the way you are, if you don't mind me asking?) It seems there are some less invasive ways to do it that might work better for you. I just tested by setting the following sysctl:

# sysctl net.ipv6.conf.all.disable_ipv6 1
net.ipv6.conf.all.disable_ipv6 = 1

After restarting MAAS, our IPv4-mapped IPv6 addresses still work fine, but no IPv6 addressing is allowed on the system.

Have you modified the kernel, or maybe passed it some additional paramters (such as `ipv6.disable=1`)? It looks like support for IPv6 sockets on your system has been completely removed/disabled.

[Mike Pontillo (mpontillo)]

Sorry, there was a typo in that command; I meant "sysctl net.ipv6.conf.all.disable_ipv6=1", for the record.

[Bruno Carvalho (brunowcs)]

I used on line grub conf ipv6.disable = 1 and sysctl net.ipv6.conf.all.disable_ipv6=1 and reboot my system, but my MAAS Already installed.

[Mike Pontillo (mpontillo)]

Thanks. We are considering making the addresses MAAS binds to more configurable, in which case you would be able to specify an IPv4-only or IPv6-only address.

Meanwhile, I think you can work around this by removing the kernel parameter to disable IPv6, and using the sysctl alone to ensure it is disabled. Hopefully that is acceptable in your scenario.

[Bruno Carvalho (brunowcs)]

Tks Mike!

[Bruno Carvalho (brunowcs)]

Mike! Do you can my help this problem? -> https://bugs.launchpad.net/maas/+bug/1459432

[Adam Vest (foxmulder2004)]

Hello,

I would simply like to update this bug report to indicate that it still affects the latest snap version 3.3. Tested by disabling IPv6 via grub, and then installing the MAAS snap and going through the quick setup documented here[1]. While the setup process appears normal and completes without issue, and while MAAS itself appears to be running according to "ps faux" output, the logs are replete with errors about IPv6 not being present, and the "maas status" command hangs.

It certainly would be nice to have the IP addresses that MAAS binds to user-tunable. If not to specific addresses, then at least maybe a toggle that just turns IPv6 on or off globally?

[trya uuum (tryauuum)]

for the people who want to know why people disable ipv6: I am too lazy to think about firewall rules for ipv6 and thus I disable it. If you think you are not using ipv6 think again, read about ipv6 link-local addresses