LibvirtConnTestCase.test_clean_shutdown_first_time segfaults

[Matt Riedemann]

http://logs.openstack.org/77/202777/13/gate/gate-nova-python27/af2fc54/console.html#_2015-08-06_12_39_20_984

note later:

{5} nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_clean_shutdown_first_time [] ... inprogress

2015-08-06 12:39:46.248 | Traceback (most recent call last):
2015-08-06 12:39:46.248 | File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/bin/subunit-trace", line 11, in <module>
2015-08-06 12:39:46.248 | sys.exit(main())
2015-08-06 12:39:46.249 | File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/os_testr/subunit_trace.py", line 328, in main
2015-08-06 12:39:46.249 | print_summary(sys.stdout, elapsed_time)
2015-08-06 12:39:46.249 | File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/os_testr/subunit_trace.py", line 268, in print_summary
2015-08-06 12:39:46.250 | num, time = worker_stats(w)
2015-08-06 12:39:46.250 | File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/os_testr/subunit_trace.py", line 242, in worker_stats
2015-08-06 12:39:46.250 | delta = tests[-1]['timestamps'][1] - tests[0]['timestamps'][0]
2015-08-06 12:39:46.251 | TypeError: unsupported operand type(s) for -: 'NoneType' and 'datetime.datetime'
2015-08-06 12:39:46.251 | ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh '

Looks like this just started happening:

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwibm92YS50ZXN0cy51bml0LnZpcnQubGlidmlydC50ZXN0X2RyaXZlci5MaWJ2aXJ0Q29ublRlc3RDYXNlLnRlc3RfY2xlYW5fc2h1dGRvd25fZmlyc3RfdGltZSBbXSAuLi4gaW5wcm9ncmVzc1wiIEFORCBidWlsZF9uYW1lOlwiZ2F0ZS1ub3ZhLXB5dGhvbjI3XCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0Mzg4NjkxNjUxNzgsIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0=

[Matt Riedemann (mriedem)]

This looks suspicious:

https://github.com/openstack/nova/blob/master/nova/tests/unit/virt/libvirt/test_driver.py#L539

[Matt Riedemann (mriedem)]

Might be good to finally get this in so we're sure that fakelibvirt is being used:

https://review.openstack.org/#/c/183706/

Since apparently there were some python-libvirt updates this week:

https://pypi.python.org/pypi/libvirt-python/1.2.18

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/210036

[Robert Collins (lifeless)]

Edited the title - subunit-trace has a separate bug here, but subunit isn't part of the actual segfault AFAICT.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/210334

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/210334
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e7396e7c6601cd9ab2aaddf82311353a5b4fefe3
Submitter: Jenkins
Branch: master

commit e7396e7c6601cd9ab2aaddf82311353a5b4fefe3
Author: Daniel P. Berrange <email address hidden>
Date: Fri Aug 7 11:57:02 2015 +0100

    libvirt: don't open connection in driver constructor

    Calling host.has_min_version triggers opening of a new
    libvirt connection. This should not ever be done in the
    constructor of LibvirtDriver class, as the 'host_init'
    method must be allowed to run first during normal Nova
    compute service startup.

    This mistake was introduced in

      commit 9a09674220a071e51fdca7911b52c0027c01ff64
      Author: Vladik Romanovsky <email address hidden>
      Date: Wed Oct 15 14:42:37 2014 -0400

        libvirt: enable virtio-net multiqueue

    and is thought to also be causing instability in the
    unit tests in the gate.

    Avoid the problem by passing the Host object instance
    into the VIF driver get_config method, instead of
    trying to pass a flag into its constructor.

    Closes-Bug: #1482230
    Change-Id: Ic940d6ffc5b7d664540363b8bcbea23c7bc7d9dd

[OpenStack Infra (hudson-openstack)]

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/210036
Reason: This didn't close the bug and danpb took care of the weird mock thing here:

https://review.openstack.org/#/c/210334/3/nova/tests/unit/virt/libvirt/test_driver.py