TestEncryptedCinderVolumes fails with 'NoneType can't be decoded'

[Matt Riedemann]

http://logs.openstack.org/19/155319/13/check/check-tempest-dsvm-full-ceph/4a14a01/logs/screen-n-cpu.txt.gz?level=TRACE#_2015-03-10_17_19_46_145

2015-03-10 17:19:46.145 ERROR oslo_messaging.rpc.dispatcher [req-72819513-908b-4210-a4c7-7f5d9ff7fd22 TestEncryptedCinderVolumes-1577051342 TestEncryptedCinderVolumes-2041152902] Exception during message handling: <type 'NoneType'> can't be decoded
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher executor_callback))
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher executor_callback)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 418, in decorated_function
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/exception.py", line 88, in wrapped
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher payload)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/exception.py", line 71, in wrapped
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 302, in decorated_function
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher pass
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 287, in decorated_function
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 330, in decorated_function
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 318, in decorated_function
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 4736, in detach_volume
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher self._detach_volume(context, instance, bdm)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/opt/stack/new/nova/nova/compute/manager.py", line 4677, in _detach_volume
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher connection_info = jsonutils.loads(bdm.connection_info)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_serialization/jsonutils.py", line 215, in loads
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher return json.loads(encodeutils.safe_decode(s, encoding), **kwargs)
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/encodeutils.py", line 33, in safe_decode
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher raise TypeError("%s can't be decoded" % type(text))
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher TypeError: <type 'NoneType'> can't be decoded
2015-03-10 17:19:46.145 30975 TRACE oslo_messaging.rpc.dispatcher

[Matt Riedemann (mriedem)]

It's not just the ceph job:

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiVGVzdEVuY3J5cHRlZENpbmRlclZvbHVtZXNcIiBBTkQgbWVzc2FnZTpcIl9kZXRhY2hfdm9sdW1lXCIgQU5EIG1lc3NhZ2U6XCJjYW5cXCd0IGJlIGRlY29kZWRcIiBBTkQgdGFnczpcInNjcmVlbi1uLWNwdS50eHRcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQyNjI1Nzg0NjczOCwibW9kZSI6IiIsImFuYWx5emVfZmllbGQiOiIifQ==

[Matt Riedemann (mriedem)]

Bug 1327218 was previously reported for the same issue.