stable/ocata nova on 16.04 (xenial)/lxd 2.0.11 can't pass gate

[Alex Kavanagh]

So the "legacy-tempest-dsvm-lxd-ovs" test is failing with stable/ocata for a very unusual set of circumstances:

On Xenial 16.04 lxd is now 2.0.11. lxd 2.0.11 grew an extra attribute on the Container called "description".

For stable/ocata, pylxd is has an upper constraint of 2.2.2.
pylxd 2.2.2 can't handle the additional attribute; the ability to ignore extra (unknown) attributes was only added in pylxd 2.2.4 due to https://github.com/lxc/pylxd/pull/230 being merged in May last year.

Therefore, the gate is broken. nova-lxd can't be patched around this as as the bug occurs between pylxd and lxd. The only real solution is to up the constraint for pylxd to 2.2.4.

I've tested this in devstack and it passes; I'm trying to reproduce the legacy-tempest-dsvm-lxd-ovs test to make sure it works in that too.

[Alex Kavanagh (ajkavanagh)]

2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [req-30924d9c-7425-4f99-94dc-e94e8d47d759 tempest-AggregatesAdminTestJSON-176150702 tempest-AggregatesAdminTestJSON-176150702] [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] Instance failed to spawn
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] Traceback (most recent call last):
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/opt/stack/new/nova/nova/compute/manager.py", line 2132, in _build_resources
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] yield resources
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/opt/stack/new/nova/nova/compute/manager.py", line 1938, in _build_and_run_instance
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] block_device_info=block_device_info)
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/opt/stack/new/nova-lxd/nova/virt/lxd/driver.py", line 444, in spawn
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] container.start(wait=True)
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/usr/local/lib/python2.7/dist-packages/pylxd/models/container.py", line 173, in start
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] wait=wait)
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/usr/local/lib/python2.7/dist-packages/pylxd/models/container.py", line 161, in _set_state
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] self.sync()
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/usr/local/lib/python2.7/dist-packages/pylxd/models/_model.py", line 154, in sync
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] setattr(self, key, val)
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] File "/usr/local/lib/python2.7/dist-packages/pylxd/models/_model.py", line 135, in __setattr__
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] return super(Model, self).__setattr__(name, value)
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7] AttributeError: 'Container' object has no attribute 'description'
2018-01-26 11:10:15.341 1262 ERROR nova.compute.manager [instance: c9bccfb3-7fe8-4146-8aec-973789e979d7]
2018-01-26 11:10:15.343 1262 INFO nova.compute.manager [req-30924d9c-7425-4f99-94dc-e94e8d47d759 tempest-AggregatesAdminTestJSON-176150702 tempest-AggregatesAdminTestJSON-176150702] [instance: c9bccfb3-7fe8-4146-8aec...

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/540437

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/540448

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/540448
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_nova/commit/?id=1aaa6b4449775a6de7bb8f4a0dab3dc69d7b2f57
Submitter: Zuul
Branch: stable/newton

commit 1aaa6b4449775a6de7bb8f4a0dab3dc69d7b2f57
Author: Jesse Pretorius <email address hidden>
Date: Fri Feb 2 16:58:50 2018 +0000

    Switch the LXD compute driver test to non-voting

    LXD 2.0.11 grew a new feature ('description' tag for a container)
    and this has been added to the stable Ubuntu Xenial 16.04 release.
    Unfortunately, pylxd 2.2.2 can't handle this extra attribute and
    crashes out. This was fixed in pylxd 2.2.4, but has not been ported
    to the 2.0.x series.

    As such, we switch the test job to non-voting until an upstream
    fix is available.

    Change-Id: Ia4f1351095023cb90d6491f4dbacaf0288f728c1
    Related-Source-Bug: https://bugs.launchpad.net/ubuntu/+source/python-pylxd/+bug/1736454
    Related-Bug: #1746068

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/540437
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=0cd3438721934773a7295f2ff9102b4e9524bf2d
Submitter: Zuul
Branch: stable/ocata

commit 0cd3438721934773a7295f2ff9102b4e9524bf2d
Author: Jesse Pretorius <email address hidden>
Date: Fri Feb 2 16:23:53 2018 +0000

    Update pylxd to 2.2.4 upper constraint as fix for lxd 2.0.11

    This overrides the upper-constraint for pylxd as a temporary
    measure to allow the nova LXD compute driver to work properly
    with recent updates to LXD.

    LXD 2.0.11 grew a new feature ('description' tag for a container)
    and this has been added to the stable Ubuntu Xenial 16.04 release.
    Unfortunately, pylxd 2.2.2 can't handle this extra attribute and
    crashes out. This was fixed in pylxd 2.2.4.

    Change-Id: I995c5ca3b012fd5275125e7a9d44e47ed08d60fb
    Related-Review: https://review.openstack.org/539174
    Related-Source-Bug: https://bugs.launchpad.net/ubuntu/+source/python-pylxd/+bug/1736454
    Related-Bug: #1746068

[Matthew Thode (prometheanfire)]

requirements fix was in https://github.com/openstack/requirements/commit/ae4a4351ada1e4c331e991ab9850a153f8dc98c8

[Alex Kavanagh (ajkavanagh)]

This was fixed in as Matthew Thode indicated.