CRITICAL nova [-] 'module' object has no attribute 'packs'

Bug #1134575 reported by Huan Zhang on 2013-02-27
30
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Undecided
Doug Hellmann
devstack
Undecided
Doug Hellmann
Ubuntu
Undecided
Unassigned

Bug Description

I got the following errors when I was deploying OpenStack with devstack.

2013-02-28 01:08:28 DEBUG nova.utils [-] backend <module 'nova.db.sqlalchemy.migration' from '/opt/stack
/nova/nova/db/sqlalchemy/migration.pyc'> from (pid=27011) __get_backend /opt/stack/nova/nova/utils.py:50
6
Command failed, please check log for more info
2013-02-28 01:09:15 CRITICAL nova [-] 'module' object has no attribute 'packs'
2013-02-28 01:09:15 TRACE nova Traceback (most recent call last):
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/bin/nova-manage", line 1404, in <module>
2013-02-28 01:09:15 TRACE nova main()
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/bin/nova-manage", line 1392, in main
2013-02-28 01:09:15 TRACE nova rpc.cleanup()
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 203,
 in cleanup
2013-02-28 01:09:15 TRACE nova return _get_impl().cleanup()
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 264,
 in _get_impl
2013-02-28 01:09:15 TRACE nova _RPCIMPL = importutils.import_module(cfg.CONF.rpc_backend)
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/importutils.py", line 58, i
n import_module
2013-02-28 01:09:15 TRACE nova __import__(import_str)
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/impl_kombu.py", line 30
, in <module>
2013-02-28 01:09:15 TRACE nova import kombu.messaging
2013-02-28 01:09:15 TRACE nova File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 15, in
 <module>
2013-02-28 01:09:15 TRACE nova from kombu.serialization import encode
2013-02-28 01:09:15 TRACE nova File "/usr/lib/python2.7/site-packages/kombu/serialization.py", line 30
0, in <module>
2013-02-28 01:09:15 TRACE nova register_msgpack()
2013-02-28 01:09:15 TRACE nova File "/usr/lib/python2.7/site-packages/kombu/serialization.py", line 28
2, in register_msgpack
2013-02-28 01:09:15 TRACE nova registry.register('msgpack', msgpack.packs, msgpack.unpacks,
2013-02-28 01:09:15 TRACE nova AttributeError: 'module' object has no attribute 'packs'
2013-02-28 01:09:15 TRACE nova
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace

Huan Zhang (hzhang) on 2013-02-27
affects: openstack-manuals → devstack

Im having exact same problem - installing latest Folsom on Ubuntu 12.04.2

Oh how I do so HATE python stacktraces... Useless.

Found my issue. Someone had installed some other packages in /usr/local/lib/python2.7/dist-packages that included the "msgpack" package, which causes the issue above.

I moved that package directory aside (did not delete it since Im not sure who put it there) and the nova packages installed successfully.

Dean Troyer (dtroyer) on 2013-04-29
Changed in devstack:
status: New → Incomplete
Brian Waldon (bcwaldon) wrote :

Actually, ceilometer needs msgpack to operate properly. It looks like kombu needs the 0.1.x version of msgpack which has that attribute 'packs' referenced in the traceback above. After installing msgpack-python==0.1.13 from pypi, I could get past a similar traceback in ceilometer-acompute startup.

Changed in devstack:
status: Incomplete → Confirmed
Changed in devstack:
assignee: nobody → Doug Hellmann (doug-hellmann)
Doug Hellmann (doug-hellmann) wrote :

Kombu was been updated to work with the new msgpack: https://github.com/celery/kombu/pull/143 a while back, but devstack installed kombu from a system package and got an old version (1.4.3 on my precise dev vm). The latest version of kombu on PyPI is 2.5.10.

Changed in devstack:
status: Confirmed → In Progress
Doug Hellmann (doug-hellmann) wrote :
Changed in ceilometer:
assignee: nobody → Doug Hellmann (doug-hellmann)
status: New → In Progress
milestone: none → havana-2

Reviewed: https://review.openstack.org/32787
Committed: http://github.com/openstack/ceilometer/commit/2d202224bc98fc882f29f00b5f97bbb7c4c720f5
Submitter: Jenkins
Branch: master

commit 2d202224bc98fc882f29f00b5f97bbb7c4c720f5
Author: Doug Hellmann <email address hidden>
Date: Wed Jun 12 16:06:18 2013 -0400

    Update requirements to fix devstack installation

    Updating kombu to a newer version also requires a new anyjson
    version. If we don't specify the versions here, some of the
    other projects install older versions that don't work together
    when running under devstack.

    Fixes bug #1134575

    Change-Id: Idcb45ca4b2c5d947b01a8b75f7906a3df6380ebe
    Signed-off-by: Doug Hellmann <email address hidden>

Changed in ceilometer:
status: In Progress → Fix Committed
Gonki (sergei-sheinin) wrote :
Download full text (3.3 KiB)

I am experiencing what appears to be the same problem:

+ /usr/local/bin/nova-manage --config-file /etc/nova/nova.conf network create private 10.0.0.0/24 1 256
2013-06-27 07:05:54.727 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2013-06-27 07:05:54.742 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db from (pid=8162) __new__ /opt/stack/nova/nova/servicegroup/api.py:61
2013-06-27 07:05:56.255 DEBUG nova.openstack.common.lockutils [req-335b63a5-c272-4f82-a285-3597539b681e None None] Got semaphore "dbapi_backend" for method "__get_backend"... from (pid=8162) inner /opt/stack/nova/nova/openstack/common/lockutils.py:190
Command failed, please check log for more info
2013-06-27 07:06:07.694 CRITICAL nova [-] 'module' object has no attribute 'packs'
2013-06-27 07:06:07.694 TRACE nova Traceback (most recent call last):
2013-06-27 07:06:07.694 TRACE nova File "/usr/local/bin/nova-manage", line 9, in <module>
2013-06-27 07:06:07.694 TRACE nova load_entry_point('nova==2013.2.a1401.gcefb051', 'console_scripts', 'nova-manage')()
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 1303, in main
2013-06-27 07:06:07.694 TRACE nova rpc.cleanup()
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 240, in cleanup
2013-06-27 07:06:07.694 TRACE nova return _get_impl().cleanup()
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 301, in _get_impl
2013-06-27 07:06:07.694 TRACE nova _RPCIMPL = importutils.import_module(CONF.rpc_backend)
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/importutils.py", line 58, in import_module
2013-06-27 07:06:07.694 TRACE nova __import__(import_str)
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/impl_kombu.py", line 30, in <module>
2013-06-27 07:06:07.694 TRACE nova import kombu.messaging
2013-06-27 07:06:07.694 TRACE nova File "/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 17, in <module>
2013-06-27 07:06:07.694 TRACE nova from .serialization import encode
2013-06-27 07:06:07.694 TRACE nova File "/usr/lib/python2.7/dist-packages/kombu/serialization.py", line 332, in <module>
2013-06-27 07:06:07.694 TRACE nova register_msgpack()
2013-06-27 07:06:07.694 TRACE nova File "/usr/lib/python2.7/dist-packages/kombu/serialization.py", line 314, in register_msgpack
2013-06-27 07:06:07.694 TRACE nova registry.register('msgpack', msgpack.packs, msgpack.unpacks,
2013-06-27 07:06:07.694 TRACE nova AttributeError: 'module' object has no attribute 'packs'
2013-06-27 07:06:07.694 TRACE nova
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace
linux@ubuntu:~/devstack$

I tried applying the patch as indicated in https://review.openstack.org/32787 but to no avail. Here are the that commands I ran before getting the error:

$ git clone git://github.com/openstack-dev/devstack.git
$ cd devstack
$ git fetch https://review.openstack.org/openstack/ceilometer refs/changes/87/32787/1 && git format-patch -1 --stdout FETCH_HEAD
...

Read more...

Thierry Carrez (ttx) on 2013-07-17
Changed in ceilometer:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2013-10-17
Changed in ceilometer:
milestone: havana-2 → 2013.2
JR (botemout) wrote :
Download full text (4.3 KiB)

I have a 4 node cluster running on ubuntu precise and, after running well for a few days of testing I now get:

root@nebula02:~# service nova-compute status
nova-compute stop/waiting
root@nebula02:~# service nova-compute start
nova-compute start/running, process 17972
root@nebula02:~# service nova-compute status
nova-compute stop/waiting

2013-10-30 10:07:35.644 17929 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2013-10-30 10:07:35.648 17929 INFO nova.manager [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-10-30 10:07:35.696 CRITICAL nova [req-212ae4d9-2206-4e8f-ae0c-4548145093ec None None] 'module' object has no attribute 'packs'
2013-10-30 10:07:35.696 17929 TRACE nova Traceback (most recent call last):
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/bin/nova-compute", line 83, in <module>
2013-10-30 10:07:35.696 17929 TRACE nova db_allowed=False)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 534, in create
2013-10-30 10:07:35.696 17929 TRACE nova db_allowed=db_allowed)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 422, in __init__
2013-10-30 10:07:35.696 17929 TRACE nova self.conductor_api.wait_until_ready(context.get_admin_context())
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 379, in wait_until_ready
2013-10-30 10:07:35.696 17929 TRACE nova self.ping(context, '1.21 GigaWatts', timeout=timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 387, in ping
2013-10-30 10:07:35.696 17929 TRACE nova return self.conductor_rpcapi.ping(context, arg, timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 102, in ping
2013-10-30 10:07:35.696 17929 TRACE nova return self.call(context, msg, version='1.22', timeout=timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line 80, in call
2013-10-30 10:07:35.696 17929 TRACE nova return rpc.call(context, self._get_topic(topic), msg, timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py", line 140, in call
2013-10-30 10:07:35.696 17929 TRACE nova return _get_impl().call(CONF, context, topic, msg, timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py", line 301, in _get_impl
2013-10-30 10:07:35.696 17929 TRACE nova _RPCIMPL = importutils.import_module(CONF.rpc_backend)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/importutils.py", line 58, in import_module
2013-10-30 10:07:35.696 17929 TRACE nova __import__(import_str)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 30, in <module>
2013-10-30 10:07:...

Read more...

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubuntu:
status: New → Confirmed
JR (botemout) wrote :

Greetings,

Sorry, but I didn't carefully read the history of this bug. As with Wyllys Ingersoll, it appears that an installation of fluentd added

/usr/local/lib/python2.7/dist-packages/msgpack

and
/usr/local/lib/python2.7/dist-packages/msgpack_python-0.4.0.egg-info

removing them results in things working again.

Dean Troyer (dtroyer) on 2013-12-13
Changed in devstack:
status: In Progress → Invalid

1, yun -y install msgpack-python

2,modif vim /usr/lib/python2.6/site-packages/kombu-1.1.3-py2.6.egg/kombu/serialization.py

278 def register_msgpack():
279 """See http://msgpack.sourceforge.net/"""
280 try:
281 import msgpack
282 #registry.register('msgpack', msgpack.packs, msgpack.unpacks,
283 registry.register('msgpack', msgpack.pack, msgpack.unpack,
284 content_type='application/x-msgpack',
285 content_encoding='binary')
286 except ImportError:
287
288 def not_available(*args, **kwargs):
289 """In case a client receives a msgpack message, but yaml
290 isn't installed."""
291 raise SerializerNotInstalled(
292 "No decoder installed for msgpack. "
293 "Install the msgpack library")
294 registry.register('msgpack', None, not_available,
295 'application/x-msgpack')

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers