Juju 2.0 Bootstrap Fails on Ubuntu Trusty Power machine.

[Prabakaran]

Hello Team,

Getting this error http://pastebin.ubuntu.com/18712357/ while bootstrapping the environment in Juju 2.0 installation on Ubuntu Trusty Power machine.

Juju 2.0 Installation Steps which i have followed is http://pastebin.ubuntu.com/18712502/

Below are log file presented under /var/log/lxd/lxd.log and /var/log/lxd/juju-*/*.log

http://paste.ubuntu.com/18713495/
http://paste.ubuntu.com/18713849/
http://paste.ubuntu.com/18713851/
http://paste.ubuntu.com/18713852/

To reproduce these error/logs install juju 2.0 on trusty power machine.

Thanks

[Cheryl Jennings (cherylj)]

Can you:

1 - Check the version of lxd installed?

2 - Make sure that lxdbr0 is configured?

Searching on the forkstart error, I see that missing lxdbr0 is a possible cause, and that lxd 2.0.3 has a fix that could help with this error (https://github.com/lxc/lxd/issues/2121)

[Prabakaran (prabacha)]

Thanks for the comment on this !!

1 - Check the version of lxd installed?

Version is 2.0.3

2 - Make sure that lxdbr0 is configured?

I tried configuring lxrbr0 by running the below commmands

sudo brctl addbr lxcbr0 ----> This command throws the error message stating "device lxcbr0 already exists; can't create bridge with the same name"

sudo ifconfig lxcbr0 10.0.3.1 netmask 255.255.255.0 up

After running the above commands i did "juju bootstrap lxd-test localhost" still i am getting the bootstrap error. Below is the error message.

"Creating Juju controller "lxd-test" on localhost/localhost
Bootstrapping model "controller"
Starting new instance for initial controller
Launching instance
ERROR failed to bootstrap model: cannot start bootstrap instance: Error calling 'lxd forkstart juju-58d450-0 /var/lib/lxd/containers /var/log/lxd/juju-58d450-0/lxc.conf': err='exit status 1'
"

Associated logs

http://paste.ubuntu.com/19345625/
http://paste.ubuntu.com/19345626/
http://paste.ubuntu.com/19345627/

Thanks

[Prabakaran (prabacha)]

Adding to the previous comment. Please find the below further more details on this issue.

1, juju version

      2.0-beta11-trusty-ppc64el

2, juju bootstrap lxd-test localhost --debug -v 2>&1 | tee /tmp/bootstrap.txt

     Log file is : http://paste.ubuntu.com/19930393/

Please look into this issue at the your earliest convenience

Thanks

[Martin Packman (gz)]

The interesting problems here all seem to be at the lxd level, from the second log in comment #2:

lxc 20160714064249.306 INFO lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:238 - changed apparmor profile to lxd-juju-47a009-0_</var/lib/lxd>
lxc 20160714064249.306 ERROR lxc_seccomp - seccomp.c:lxc_seccomp_load:615 - Error loading the seccomp policy
lxc 20160714024249.306 ERROR lxc_sync - sync.c:__sync_wait:57 - An error occurred in another process (expected sequence number 5)
lxc 20160714024249.306 ERROR lxc_start - start.c:__lxc_start:1353 - failed to spawn 'juju-47a009-0'
lxc 20160714024249.306 INFO lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'juju-47a009-0', config section 'lxc'
lxc 20160714024249.812 INFO lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/bin/lxd callhook /var/lib/lxd 3 stop' for container 'juju-47a009-0', config section 'lxc'
lxc 20160714024249.963 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response
lxc 20160714024249.964 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_cgroup failed to receive response

[Stéphane Graber (stgraber)]

Oh, you said on a trusty host. Then that's your problem.

The release kernel for trusty (3.13) isn't capable of seccomp and so containers just fail to start.

Upgrade to the 4.4 kernel and things should start working again.

[Matt Bruzek (mbruzek)]

Since this seems to be a LXD problem please pastebin the following commands:

sudo lxc list
sudo lxc remote add images images.linuxcontainers.org
sudo lxc launch images:ubuntu/trusty/ppc64el ubuntu-test
sudo lxc list
sudo lxc stop ubuntu-test
sudo lxc delete ubuntu-test

If those commands fail, pastebin:
   /var/lib/lxd/security/apparmor/profiles/lxd-ubuntu-test
  "grep 'apparmor' /var/log/syslog"
  /var/log/lxd/ubuntu-test/lxc.log

[Stéphane Graber (stgraber)]

3.16 (lts-utopic) and 3.19 (lts-vivid) also are lacking seccomp support, 4.2 (lts-wily) has it I believe but it wasn't tested. 4.4 is what we test ppc64el with and has been confirmed to have seccomp support.

We have no plans to support other kernels on ppc64el as it'd mean turning off a critical security feature, making our security story inconsistent depending on kernel version and architecture.

We have a change coming in the next LXD bugfix release which will better surface such LXC errors and should help diagnosing that kind of thing in the future.

[Stéphane Graber (stgraber)]

Marking invalid for LXD as we don't support running on kernels without seccomp support. Upgrading to linux-generic-lts-xenial should fix this issue for you (confirmed on a test system).