failed to bridge devices: bridge activaction error: bridge activation failed: Killed old client process

[james beedy]

hitting this on multiple machines - "failed to start machine 2/lxd/0 (failed to bridge devices: bridge activaction error: bridge activation failed: Killed old client process"

$ juju status --format yaml | pastebinit
http://paste.ubuntu.com/26107129/

[james beedy (jamesbeedy)]

controller logs

[james beedy (jamesbeedy)]

maas region+rack logs

[james beedy (jamesbeedy)]

something to note here, I did not hit this issue unit I deploy the stack as a bundle http://paste.ubuntu.com/26107347/.

When casually deploying the same stack ^ with singleton commands from the cli I don't experience this issue, but I can replicate it by deploying the bundle all in one go.

[james beedy (jamesbeedy)]

*did not hit this issue until I deployed the stack as a bundle

[james beedy (jamesbeedy)]

I just hit this by deploy just a few simple things by way of running commands on the cli.

$ juju status --format yaml | pastebinit
http://paste.ubuntu.com/26113313/

$ juju status | pastebinit
http://paste.ubuntu.com/26113315/

[james beedy (jamesbeedy)]

from a machine experiencing the issue

sudo cat /var/log/juju/machine-2.log | pastebinit
http://paste.ubuntu.com/26113348/

[james beedy (jamesbeedy)]

just sent off a new deploy, whats odd is that I see the error in the logs, but this time, my lxd containers all deployed successfully

debug-log http://paste.ubuntu.com/26113536/

$ juju status | pastebinit
http://paste.ubuntu.com/26113564/

$ juju status --format yaml | pastebinit
http://paste.ubuntu.com/26113566/

[Mike Pontillo (mpontillo)]

Do I assume correctly that you didn't do anything specific in MAAS to create the bridges?

Usually Juju will create bridges on its own; MAAS supports the creation of bridge interfaces upon deploy (thus allowing Juju to instruct MAAS to create them), but to my knowledge (correct me if I'm wrong) Juju is still doing the work itself.

[james beedy (jamesbeedy)]

@mpontillo that is correct my interface definitions in maas resemble screen shot attached

[John A Meinel (jameinel)]

According to this:
http://www.microhowto.info/howto/configure_an_ethernet_interface_as_a_vlan_trunk_on_debian.html

It seems to say that maybe there was a problem finding the underlying device.
I wonder if the issue is that the underlying enp4s0f0 is left "unconfigured"?

The other thing you could potentially do is try to create a bridge in MAAS, which should then already be configured by the time Juju comes along, which should mean that Juju uses whatever bridge you've already declared.

The main reason that Juju doesn't pre-configure MAAS machines is because
a) I'm not sure if configuring it ahead of time means that MAAS would then forever think you want exactly those bridges on that machine. Meaning if you then release the machine, and later wanted to use it for a different purpose, you would still end up with the bridges. People have very specifically asked for us to only bridge the interfaces that expressly need them. So if a given machine doesn't run any containers, then they don't want any bridges on that machine.
b) You can always ask after provisioning the base machine to then add a container at any point. Which means Juju already needed all the logic to handle someone asking for a container 2-months after deploying the original machine. Better to use that code path regularly.

However, Juju *does* use bridges that already exist, so if you have a reason that you don't want to wait for dynamic bridging, and you know how the machines will be used, you can go craft that in MAAS and Juju will just use those settings that were already defined.

[John A Meinel (jameinel)]

I don't think we quite have enough information to understand what is going wrong here.

[james beedy (jamesbeedy)]

@jameinel I've tried creating a bridge on the interface in MAAS prior to deploying the node http://paste.ubuntu.com/26177785/

I still get the same error it seems http://paste.ubuntu.com/26177838/

ubuntu@os-ceph-00:~$ cat /etc/network/interfaces.d/50-cloud-init.cfg | http://paste.ubuntu.com/26177856/

[james beedy (jamesbeedy)]

I have feeling this is the same bug as https://bugs.launchpad.net/juju/+bug/1737640

[james beedy (jamesbeedy)]

this duplicate is fixed with whats in proposed per https://bugs.launchpad.net/juju/+bug/1737640