Seen in a new nova-lxc job running on ubuntu 18.04 devstack master (train):
https://logs.opendev.org/24/676024/5/experimental/nova-lxc/9c06394/controller/logs/screen-n-cpu.txt.gz#_Aug_13_23_16_20_055037
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: ERROR nova.virt.libvirt.guest [None req-1e0d72dd-6dfb-4040-b900-37dfe43e0ef6 tempest-AggregatesAdminTestJSON-952562850 tempest-AggregatesAdminTestJSON-952562850] Error launching a defined domain with XML: <domain type='lxc'>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <name>instance-00000001</name>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <uuid>929b9a5e-dfcb-428a-a2a7-55ab5bf76c9b</uuid>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <metadata>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0">
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:package version="19.1.0"/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:name>tempest-AggregatesAdminTestJSON-server-302754080</nova:name>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:creationTime>2019-08-13 23:16:17</nova:creationTime>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:flavor name="m1.nano">
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:memory>64</nova:memory>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:disk>1</nova:disk>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:swap>0</nova:swap>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:ephemeral>0</nova:ephemeral>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:vcpus>1</nova:vcpus>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </nova:flavor>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:owner>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:user uuid="f2220c686d3e4903940bcd1a1e2efc0b">tempest-AggregatesAdminTestJSON-952562850</nova:user>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:project uuid="8ae0fe76b0e54829971c2cb44f3770ba">tempest-AggregatesAdminTestJSON-952562850</nova:project>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </nova:owner>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <nova:root type="image" uuid="eab0eae0-2961-4b19-9fd7-4f0e0e0a2fa8"/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </nova:instance>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </metadata>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <memory unit='KiB'>65536</memory>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <currentMemory unit='KiB'>65536</currentMemory>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <vcpu placement='static'>1</vcpu>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <cputune>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <shares>1024</shares>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </cputune>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <resource>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <partition>/machine</partition>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </resource>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <os>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <type arch='x86_64'>exe</type>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <init>/sbin/init</init>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <cmdline>console=tty0 console=ttyS0 console=hvc0</cmdline>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </os>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <clock offset='utc'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <on_poweroff>destroy</on_poweroff>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <on_reboot>restart</on_reboot>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <on_crash>destroy</on_crash>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <devices>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <emulator>/usr/lib/libvirt/libvirt_lxc</emulator>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <filesystem type='mount' accessmode='passthrough'>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <source dir='/opt/stack/data/nova/instances/929b9a5e-dfcb-428a-a2a7-55ab5bf76c9b/rootfs'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <target dir='/'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </filesystem>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <interface type='bridge'>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <mac address='fa:16:3e:e0:14:6c'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <source bridge='br-int'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <virtualport type='openvswitch'>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <parameters interfaceid='5a189d67-24c6-4a33-9b8e-5985acf85dd7'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </virtualport>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <target dev='tap5a189d67-24'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <mtu size='1400'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </interface>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <console type='pty'>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <log file='/opt/stack/data/nova/instances/929b9a5e-dfcb-428a-a2a7-55ab5bf76c9b/console.log' append='off'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: <target type='lxc' port='0'/>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </console>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </devices>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: </domain>
Aug 13 23:16:20.055037 ubuntu-bionic-ovh-bhs1-0010107422 nova-compute[26642]: : libvirt.libvirtError: internal error: guest failed to start: Failure in libvirt_lxc startup: cannot find init path '/sbin/init' relative to container root: No such file or directory
It looks like misconfig in devstack for lxc images. We could potentially workaround this by configuring the image used in devstack with lxc to set the os_command_line image property to point at systemd rather than init:
https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L63-L67
"os_command_line": {
"title": "Kernel Command Line",
"description": "The kernel command line to be used by the libvirt driver, instead of the default. For linux containers (LXC), the value is used as arguments for initialization. This key is valid only for Amazon kernel, ramdisk, or machine images (aki, ari, or ami).",
"type": "string"
},
Seems the nova-lxc was only running 3 times in its total lifetime, so this doesn't seem to be relevant any longer.