test_create_multiple_servers fails on master with venv

Bug #861440 reported by Mark McLoughlin on 2011-09-28
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Brian Lamar
kombu (Ubuntu)
Dave Walker

Bug Description

Using venv, I'm seeing test_create_multiple_servers fail on master

The test was introduced by:

  commit 0fab78825ef06310926181f6f97d377058b56b97
  Author: Chris Behrens <email address hidden>
  Date: Sat Sep 24 23:49:36 2011 +0000

    compute_api create*() and schedulers refactoring

The error is:

2011-09-28 14:03:24,538 INFO nova.rpc [-] Connected to AMQP server on localhost:5672
2011-09-28 14:03:24,544 ERROR nova.rpc [-] Exception during message handling
(nova.rpc): TRACE: Traceback (most recent call last):
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 620, in _process_data
(nova.rpc): TRACE: rval = node_func(context=ctxt, **node_args)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/exception.py", line 98, in wrapped
(nova.rpc): TRACE: return f(*args, **kw)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/compute/manager.py", line 119, in decorated_function
(nova.rpc): TRACE: function(self, context, instance_id, *args, **kwargs)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/compute/manager.py", line 512, in terminate_instance
(nova.rpc): TRACE: self._delete_instance(context, instance_id)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/compute/manager.py", line 493, in _delete_instance
(nova.rpc): TRACE: self._shutdown_instance(context, instance_id, 'Terminating')
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/compute/manager.py", line 474, in _shutdown_instance
(nova.rpc): TRACE: network_info = self._get_instance_nw_info(context, instance)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/compute/manager.py", line 241, in _get_instance_nw_info
(nova.rpc): TRACE: instance)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/network/api.py", line 210, in get_instance_nw_info
(nova.rpc): TRACE: 'args': args})
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/__init__.py", line 45, in call
(nova.rpc): TRACE: return get_impl().call(context, topic, msg)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 738, in call
(nova.rpc): TRACE: rv = list(rv)
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 705, in __iter__
(nova.rpc): TRACE: self.done()
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 685, in done
(nova.rpc): TRACE: self._connection.close()
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 570, in close
(nova.rpc): TRACE: self._done()
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 547, in _done
(nova.rpc): TRACE: self.connection.reset()
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/nova/rpc/impl_kombu.py", line 382, in reset
(nova.rpc): TRACE: self.channel.close()
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/.nova-venv/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 560, in close
(nova.rpc): TRACE: self._qos.restore_unacked_once()
(nova.rpc): TRACE: File "/home/markmc/projects/openstack/git/nova/.nova-venv/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 165, in restore_unacked_once
(nova.rpc): TRACE: assert not state
(nova.rpc): TRACE: AssertionError

Related branches

Chris Behrens (cbehrens) wrote :

It's a bug in kombu memory transport used for tests. Need to downgrade to version 1.0.4, which is the same version that's in our ppa until we can get a fix into newer kombu... or.. we stop using kombu's memory transport for tests in favor of a fakerabbit port.

Note that other tests will randomly fail with the same thing... some snapshot tests.

Lorin Hochstein (lorinh) on 2011-09-29
Changed in nova:
status: New → Fix Committed
Dave Walker (davewalker) wrote :

@Lorin, you've marked this Fix Committed for a nova task, can you comment where this is? I couldn't find the commit.


Dave Walker (davewalker) on 2011-10-18
Changed in kombu (Ubuntu):
milestone: none → precise-alpha-1
assignee: nobody → Dave Walker (davewalker)
importance: Undecided → Medium
status: New → Confirmed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package kombu - 1.2.1-1ubuntu1

kombu (1.2.1-1ubuntu1) precise; urgency=low

  * debian/patches/remove_assert_virtual_transport_restore_unacked_once.
    patch: Disabled seemingly invalid assert in virtual transport which
    was causing nova testsuite to fail. Pending better solution.
    - LP: #861440
 -- Dave Walker (Daviey) <email address hidden> Tue, 18 Oct 2011 15:16:16 +0100

Changed in kombu (Ubuntu):
status: Confirmed → Fix Released
Lorin Hochstein (lorinh) wrote :


I think it's eb4bd86f.

Lorin Hochstein (lorinh) wrote :

Here it is in Github: https://github.com/openstack/nova/commit/eb4bd86f

At the time, I didn't realize the Gerrit marks the bugs off as committed when they're linked, so I just manually set this one to fix committed when I found that it fixed the issue I was having.

Thierry Carrez (ttx) on 2011-11-09
Changed in nova:
milestone: none → essex-1
Thierry Carrez (ttx) on 2011-11-17
Changed in nova:
status: Fix Committed → Fix Released
Mark McLoughlin (markmc) on 2012-04-03
Changed in nova:
assignee: nobody → Brian Lamar (blamar)
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: essex-1 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers