CRITICAL nova [-] 'module' object has no attribute 'packs'

Bug #1134575 reported by Huan Zhang
30
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Ceilometer
Fix Released
Undecided
Doug Hellmann
devstack
Invalid
Undecided
Doug Hellmann
Ubuntu
Confirmed
Undecided
Unassigned

Bug Description

I got the following errors when I was deploying OpenStack with devstack.

2013-02-28 01:08:28 DEBUG nova.utils [-] backend <module 'nova.db.sqlalchemy.migration' from '/opt/stack
/nova/nova/db/sqlalchemy/migration.pyc'> from (pid=27011) __get_backend /opt/stack/nova/nova/utils.py:50
6
Command failed, please check log for more info
2013-02-28 01:09:15 CRITICAL nova [-] 'module' object has no attribute 'packs'
2013-02-28 01:09:15 TRACE nova Traceback (most recent call last):
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/bin/nova-manage", line 1404, in <module>
2013-02-28 01:09:15 TRACE nova main()
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/bin/nova-manage", line 1392, in main
2013-02-28 01:09:15 TRACE nova rpc.cleanup()
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 203,
 in cleanup
2013-02-28 01:09:15 TRACE nova return _get_impl().cleanup()
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 264,
 in _get_impl
2013-02-28 01:09:15 TRACE nova _RPCIMPL = importutils.import_module(cfg.CONF.rpc_backend)
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/importutils.py", line 58, i
n import_module
2013-02-28 01:09:15 TRACE nova __import__(import_str)
2013-02-28 01:09:15 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/impl_kombu.py", line 30
, in <module>
2013-02-28 01:09:15 TRACE nova import kombu.messaging
2013-02-28 01:09:15 TRACE nova File "/usr/lib/python2.7/site-packages/kombu/messaging.py", line 15, in
 <module>
2013-02-28 01:09:15 TRACE nova from kombu.serialization import encode
2013-02-28 01:09:15 TRACE nova File "/usr/lib/python2.7/site-packages/kombu/serialization.py", line 30
0, in <module>
2013-02-28 01:09:15 TRACE nova register_msgpack()
2013-02-28 01:09:15 TRACE nova File "/usr/lib/python2.7/site-packages/kombu/serialization.py", line 28
2, in register_msgpack
2013-02-28 01:09:15 TRACE nova registry.register('msgpack', msgpack.packs, msgpack.unpacks,
2013-02-28 01:09:15 TRACE nova AttributeError: 'module' object has no attribute 'packs'
2013-02-28 01:09:15 TRACE nova
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace

Tags: devstack
Huan Zhang (hzhang)
affects: openstack-manuals → devstack
Revision history for this message
Wyllys Ingersoll (wyllys66) wrote :

Im having exact same problem - installing latest Folsom on Ubuntu 12.04.2

Oh how I do so HATE python stacktraces... Useless.

Revision history for this message
Wyllys Ingersoll (wyllys66) wrote :

Found my issue. Someone had installed some other packages in /usr/local/lib/python2.7/dist-packages that included the "msgpack" package, which causes the issue above.

I moved that package directory aside (did not delete it since Im not sure who put it there) and the nova packages installed successfully.

Dean Troyer (dtroyer)
Changed in devstack:
status: New → Incomplete
Revision history for this message
Brian Waldon (bcwaldon) wrote :

Actually, ceilometer needs msgpack to operate properly. It looks like kombu needs the 0.1.x version of msgpack which has that attribute 'packs' referenced in the traceback above. After installing msgpack-python==0.1.13 from pypi, I could get past a similar traceback in ceilometer-acompute startup.

Changed in devstack:
status: Incomplete → Confirmed
Changed in devstack:
assignee: nobody → Doug Hellmann (doug-hellmann)
Revision history for this message
Doug Hellmann (doug-hellmann) wrote :

Kombu was been updated to work with the new msgpack: https://github.com/celery/kombu/pull/143 a while back, but devstack installed kombu from a system package and got an old version (1.4.3 on my precise dev vm). The latest version of kombu on PyPI is 2.5.10.

Changed in devstack:
status: Confirmed → In Progress
Revision history for this message
Doug Hellmann (doug-hellmann) wrote :
Revision history for this message
Doug Hellmann (doug-hellmann) wrote :
Changed in ceilometer:
assignee: nobody → Doug Hellmann (doug-hellmann)
status: New → In Progress
milestone: none → havana-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/32787

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (master)

Reviewed: https://review.openstack.org/32787
Committed: http://github.com/openstack/ceilometer/commit/2d202224bc98fc882f29f00b5f97bbb7c4c720f5
Submitter: Jenkins
Branch: master

commit 2d202224bc98fc882f29f00b5f97bbb7c4c720f5
Author: Doug Hellmann <email address hidden>
Date: Wed Jun 12 16:06:18 2013 -0400

    Update requirements to fix devstack installation

    Updating kombu to a newer version also requires a new anyjson
    version. If we don't specify the versions here, some of the
    other projects install older versions that don't work together
    when running under devstack.

    Fixes bug #1134575

    Change-Id: Idcb45ca4b2c5d947b01a8b75f7906a3df6380ebe
    Signed-off-by: Doug Hellmann <email address hidden>

Changed in ceilometer:
status: In Progress → Fix Committed
Revision history for this message
Gonki (sergei-sheinin) wrote :
Download full text (3.3 KiB)

I am experiencing what appears to be the same problem:

+ /usr/local/bin/nova-manage --config-file /etc/nova/nova.conf network create private 10.0.0.0/24 1 256
2013-06-27 07:05:54.727 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2013-06-27 07:05:54.742 DEBUG nova.servicegroup.api [-] ServiceGroup driver defined as an instance of db from (pid=8162) __new__ /opt/stack/nova/nova/servicegroup/api.py:61
2013-06-27 07:05:56.255 DEBUG nova.openstack.common.lockutils [req-335b63a5-c272-4f82-a285-3597539b681e None None] Got semaphore "dbapi_backend" for method "__get_backend"... from (pid=8162) inner /opt/stack/nova/nova/openstack/common/lockutils.py:190
Command failed, please check log for more info
2013-06-27 07:06:07.694 CRITICAL nova [-] 'module' object has no attribute 'packs'
2013-06-27 07:06:07.694 TRACE nova Traceback (most recent call last):
2013-06-27 07:06:07.694 TRACE nova File "/usr/local/bin/nova-manage", line 9, in <module>
2013-06-27 07:06:07.694 TRACE nova load_entry_point('nova==2013.2.a1401.gcefb051', 'console_scripts', 'nova-manage')()
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/cmd/manage.py", line 1303, in main
2013-06-27 07:06:07.694 TRACE nova rpc.cleanup()
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 240, in cleanup
2013-06-27 07:06:07.694 TRACE nova return _get_impl().cleanup()
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 301, in _get_impl
2013-06-27 07:06:07.694 TRACE nova _RPCIMPL = importutils.import_module(CONF.rpc_backend)
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/importutils.py", line 58, in import_module
2013-06-27 07:06:07.694 TRACE nova __import__(import_str)
2013-06-27 07:06:07.694 TRACE nova File "/opt/stack/nova/nova/openstack/common/rpc/impl_kombu.py", line 30, in <module>
2013-06-27 07:06:07.694 TRACE nova import kombu.messaging
2013-06-27 07:06:07.694 TRACE nova File "/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 17, in <module>
2013-06-27 07:06:07.694 TRACE nova from .serialization import encode
2013-06-27 07:06:07.694 TRACE nova File "/usr/lib/python2.7/dist-packages/kombu/serialization.py", line 332, in <module>
2013-06-27 07:06:07.694 TRACE nova register_msgpack()
2013-06-27 07:06:07.694 TRACE nova File "/usr/lib/python2.7/dist-packages/kombu/serialization.py", line 314, in register_msgpack
2013-06-27 07:06:07.694 TRACE nova registry.register('msgpack', msgpack.packs, msgpack.unpacks,
2013-06-27 07:06:07.694 TRACE nova AttributeError: 'module' object has no attribute 'packs'
2013-06-27 07:06:07.694 TRACE nova
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace
linux@ubuntu:~/devstack$

I tried applying the patch as indicated in https://review.openstack.org/32787 but to no avail. Here are the that commands I ran before getting the error:

$ git clone git://github.com/openstack-dev/devstack.git
$ cd devstack
$ git fetch https://review.openstack.org/openstack/ceilometer refs/changes/87/32787/1 && git format-patch -1 --stdout FETCH_HEAD
...

Read more...

Thierry Carrez (ttx)
Changed in ceilometer:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ceilometer:
milestone: havana-2 → 2013.2
Revision history for this message
JR (botemout) wrote :
Download full text (4.3 KiB)

I have a 4 node cluster running on ubuntu precise and, after running well for a few days of testing I now get:

root@nebula02:~# service nova-compute status
nova-compute stop/waiting
root@nebula02:~# service nova-compute start
nova-compute start/running, process 17972
root@nebula02:~# service nova-compute status
nova-compute stop/waiting

2013-10-30 10:07:35.644 17929 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2013-10-30 10:07:35.648 17929 INFO nova.manager [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-10-30 10:07:35.696 CRITICAL nova [req-212ae4d9-2206-4e8f-ae0c-4548145093ec None None] 'module' object has no attribute 'packs'
2013-10-30 10:07:35.696 17929 TRACE nova Traceback (most recent call last):
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/bin/nova-compute", line 83, in <module>
2013-10-30 10:07:35.696 17929 TRACE nova db_allowed=False)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 534, in create
2013-10-30 10:07:35.696 17929 TRACE nova db_allowed=db_allowed)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 422, in __init__
2013-10-30 10:07:35.696 17929 TRACE nova self.conductor_api.wait_until_ready(context.get_admin_context())
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 379, in wait_until_ready
2013-10-30 10:07:35.696 17929 TRACE nova self.ping(context, '1.21 GigaWatts', timeout=timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 387, in ping
2013-10-30 10:07:35.696 17929 TRACE nova return self.conductor_rpcapi.ping(context, arg, timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 102, in ping
2013-10-30 10:07:35.696 17929 TRACE nova return self.call(context, msg, version='1.22', timeout=timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line 80, in call
2013-10-30 10:07:35.696 17929 TRACE nova return rpc.call(context, self._get_topic(topic), msg, timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py", line 140, in call
2013-10-30 10:07:35.696 17929 TRACE nova return _get_impl().call(CONF, context, topic, msg, timeout)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py", line 301, in _get_impl
2013-10-30 10:07:35.696 17929 TRACE nova _RPCIMPL = importutils.import_module(CONF.rpc_backend)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/importutils.py", line 58, in import_module
2013-10-30 10:07:35.696 17929 TRACE nova __import__(import_str)
2013-10-30 10:07:35.696 17929 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py", line 30, in <module>
2013-10-30 10:07:...

Read more...

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ubuntu:
status: New → Confirmed
Revision history for this message
JR (botemout) wrote :

Greetings,

Sorry, but I didn't carefully read the history of this bug. As with Wyllys Ingersoll, it appears that an installation of fluentd added

/usr/local/lib/python2.7/dist-packages/msgpack

and
/usr/local/lib/python2.7/dist-packages/msgpack_python-0.4.0.egg-info

removing them results in things working again.

Dean Troyer (dtroyer)
Changed in devstack:
status: In Progress → Invalid
Revision history for this message
wangqiang.sheng (wangqiangsheng) wrote :

1, yun -y install msgpack-python

2,modif vim /usr/lib/python2.6/site-packages/kombu-1.1.3-py2.6.egg/kombu/serialization.py

278 def register_msgpack():
279 """See http://msgpack.sourceforge.net/"""
280 try:
281 import msgpack
282 #registry.register('msgpack', msgpack.packs, msgpack.unpacks,
283 registry.register('msgpack', msgpack.pack, msgpack.unpack,
284 content_type='application/x-msgpack',
285 content_encoding='binary')
286 except ImportError:
287
288 def not_available(*args, **kwargs):
289 """In case a client receives a msgpack message, but yaml
290 isn't installed."""
291 raise SerializerNotInstalled(
292 "No decoder installed for msgpack. "
293 "Install the msgpack library")
294 registry.register('msgpack', None, not_available,
295 'application/x-msgpack')

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.