I'm trying to set up armhf testing on an arm64 host, as that's what we have in Scalingstack (no armhf images yet). The host is Ubuntu 15.10, with lxd 0.20-0ubuntu4.1 (no PPA).
$ uname -a
Linux arm64-lxd-test 4.2.0-18-generic #22-Ubuntu SMP Fri Nov 6 19:56:51 UTC 2015 aarch64 aarch64 aarch64 GNU/Linux
$ lxc image list | grep arm
| ubuntu/xenial/armhf | a406edc85653 | no | ubuntu xenial armv7l (default) (20151202_04:37) | armv7l | 63.68MB | Dec 2, 2015 at 1:23pm (UTC) |
$ lxc launch ubuntu/xenial/armhf x1
Starting the container throws no error, and with debugging I don't see anything bad:
$ lxc start x1 --debug --verbose
DBUG[12-02|13:36:56] Fingering the daemon
DBUG[12-02|13:36:56] Raw response: {"type":"sync","status":"Success","status_code":200,"metadata":{"api_compat":1,"auth":"trusted","config":{"core.https_address":"10.43.41.223","images.remote_cache_expiry":"10"},"environment":{"addresses":["10.43.41.223"],"architectures":[4,3],"driver":"lxc","driver_version":"1.1.4","kernel":"Linux","kernel_architecture":"aarch64","kernel_version":"4.2.0-18-generic","server":"lxd","server_pid":1339,"server_version":"0.20","storage":"dir","storage_version":""}}}
DBUG[12-02|13:36:56] Pong received
DBUG[12-02|13:36:56] Raw response: {"type":"sync","status":"Success","status_code":200,"metadata":{"architecture":0,"config":{"volatile.base_image":"a406edc85653e7b3232ea1ae77e35b67dd42574cb4c7335e9b586a6b4ad6223c","volatile.eth0.hwaddr":"00:16:3e:38:aa:2c","volatile.last_state.idmap":"[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":100000,\"Nsid\":0,\"Maprange\":65536},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":100000,\"Nsid\":0,\"Maprange\":65536}]"},"devices":{},"ephemeral":false,"expanded_config":{"volatile.base_image":"a406edc85653e7b3232ea1ae77e35b67dd42574cb4c7335e9b586a6b4ad6223c","volatile.eth0.hwaddr":"00:16:3e:38:aa:2c","volatile.last_state.idmap":"[{\"Isuid\":true,\"Isgid\":false,\"Hostid\":100000,\"Nsid\":0,\"Maprange\":65536},{\"Isuid\":false,\"Isgid\":true,\"Hostid\":100000,\"Nsid\":0,\"Maprange\":65536}]"},"expanded_devices":{"eth0":{"hwaddr":"00:16:3e:38:aa:2c","nictype":"bridged","parent":"lxcbr0","type":"nic"}},"name":"x1","profiles":["default"],"status":{"status":"Stopped","status_code":102,"init":0,"ips":null}}}
DBUG[12-02|13:36:56] Putting {"action":"start","force":false,"timeout":-1}
to http://unix.socket/1.0/containers/x1/state
DBUG[12-02|13:36:56] Raw response: {"type":"async","status":"OK","status_code":100,"operation":"/1.0/operations/f17b8722-1573-4af8-a365-bc450bce6654","resources":null,"metadata":null}
DBUG[12-02|13:36:56] 1.0/operations/f17b8722-1573-4af8-a365-bc450bce6654/wait
DBUG[12-02|13:36:57] Raw response: {"type":"sync","status":"Success","status_code":200,"metadata":{"created_at":"2015-12-02T13:36:56.76183Z","updated_at":"2015-12-02T13:36:57.059047Z","status":"Success","status_code":200,"resources":null,"metadata":null,"may_cancel":false}}
But the container is not running afterwards. I'm attaching /var/log/lxd/x1/lxc.log, but the most interesting bits are several
WARN lxc_cgmanager - cgmanager.c:cgm_get:993 - do_cgm_get exited with error
and
NOTICE lxc_start - start.c:post_start:1265 - '/sbin/init' started with pid '2028'
WARN lxc_start - start.c:signal_handler:310 - invalid pid for SIGCHLD
DEBUG lxc_commands - commands.c:lxc_cmd_handler:893 - peer has disconnected
DEBUG lxc_commands - commands.c:lxc_cmd_handler:893 - peer has disconnected
DEBUG lxc_commands - commands.c:lxc_cmd_get_state:579 - 'x1' is in 'RUNNING' state
DEBUG lxc_start - start.c:signal_handler:314 - container init process exited
cgmanager.service itself is active and running, though.
Is there some way to get a console for this, like we used to have with "lxc-start -n foo -F"?
Some more obvservations:
- I get exactly the same failure with lxc launch'ing a trusty armhf instance.
- arm64 lxd images work fine (tested trusty and wily, there are no xenial ones yet)
So I went down a level and tried with LXC:
sudo lxc-create -n x1armhf -t ubuntu -- -r xenial -a armhf
This also fails, but with some more info:
$ sudo lxc-start -n x1armhf -F -l debug -o /dev/stderr post_start: 1265 - '/sbin/init' started with pid '13393' signal_ handler: 310 - invalid pid for SIGCHLD signal_ handler: 314 - container init process exited __lxc_start: 1207 - Container violated its seccomp policy __lxc_start: 1215 - Pushing physical nics back to host namespace __lxc_start: 1218 - Tearing down virtual network devices used by container lxc_delete_ network: 2939 - failed to remove interface '(null)' lxc_error_ set_and_ log:55 - child <13393> ended on signal (31) lxc_delete_ network: 2939 - failed to remove interface '(null)'
[..]
lxc-start 1449065480.085 NOTICE lxc_start - start.c:start:1254 - exec'ing '/sbin/init'
lxc-start 1449065480.085 NOTICE lxc_start - start.c:
lxc-start 1449065480.085 WARN lxc_start - start.c:
lxc-start 1449065480.086 DEBUG lxc_start - start.c:
lxc-start 1449065480.086 DEBUG lxc_start - start.c:
lxc-start 1449065480.086 DEBUG lxc_start - start.c:
lxc-start 1449065480.086 DEBUG lxc_start - start.c:
lxc-start 1449065480.086 WARN lxc_conf - conf.c:
lxc-start 1449065480.092 INFO lxc_error - error.c:
lxc-start 1449065480.093 WARN lxc_conf - conf.c:
and then it exits again (with code 0!), and there is no container running. Not sure if the "violated its seccomp policy" bit is interesting?
So one further step down: I directly downloaded and unpacked https:/ /images. linuxcontainers .org/images/ ubuntu/ xenial/ armhf/default/ 20151202_ 04:37/lxd. tar.xz:
$ sudo tar xpf lxd.tar.xz architecture
$ sudo chroot rootfs/
# dpkg --print-
armhf
nspawn fails too, with a different error message:
$ sudo systemd-nspawn -b -D rootfs/ rootfs. rootfs/ sys/fs/ selinux: Read-only file system rootfs/ sys/fs/ selinux: Read-only file system
Spawning container rootfs on /home/ubuntu/
Press ^] three times within 1s to kill container.
Failed to create directory /home/ubuntu/
Failed to create directory /home/ubuntu/
/etc/localtime is not a symlink, not updating container timezone.
Container rootfs terminated by signal SYS.
In syslog I'm getting seccomp errors (from LXC and nspawn):
Dec 02 14:11:57 arm64-lxd-test audit[13536]: SECCOMP auid=1000 uid=0 gid=0 ses=1 pid=13536 comm="init" exe="/lib/ systemd/ systemd" sig=31 arch=40000028 syscall=45 compat=1 ip=0xf763abd6 code=0x0 systemd/ systemd" sig=31 arch=40000028 syscall=45 compat=1 ip=0xf718fbd6 code=0x0
Dec 02 14:15:03 arm64-lxd-test audit[25812]: SECCOMP auid=4294967295 uid=0 gid=0 ses=4294967295 pid=25812 comm="systemd" exe="/lib/