'module' object has no attribute 'is_admin' when retyping

[Peter Wang]

When i do a volume retype, following error occurred:

==============================================
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher [req-ee7f46a7-993b-40e0-9724-0779f46dcb11 a63e97c42c024ecfbdd08f5c507372e7 80cb188ebf8a46ba837392acc3f3704f - - -] Exception during message handling: 'module' object has no attribute 'is_admin'
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/volume/manager.py", line 2214, in retype
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher new_reservations, status_update)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/volume/manager.py", line 2210, in retype
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher new_type_id=new_type_id)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/volume/manager.py", line 1819, in migrate_volume
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher volume = objects.Volume.get_by_id(context, volume_id)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 180, in wrapper
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher result = fn(cls, context, *args, **kwargs)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/objects/base.py", line 93, in get_by_id
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher orm_obj = db.get_by_id(context, model, id, *args, **kwargs)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/db/api.py", line 1092, in get_by_id
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher return IMPL.get_by_id(context, model, id, *args, **kwargs)
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 188, in wrapper
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher if not is_admin_context(args[0]) and not is_user_context(args[0]):
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/cinder/cinder/db/sqlalchemy/api.py", line 121, in is_admin_context
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher return context.is_admin
2016-01-21 04:15:56.308 26501 ERROR oslo_messaging.rpc.dispatcher AttributeError: 'module' object has no attribute 'is_admin'

[Mitsuhiro Tanino (mitsuhiro-tanino)]

Seems same bug was reported and resolved but let me double check whither latest master doesn't have a problem.

https://review.openstack.org/#/c/257667/
https://bugs.launchpad.net/cinder/+bug/1530772

[Mitsuhiro Tanino (mitsuhiro-tanino)]

I tried to reproduce this bug but the bug seems already fixed.

http://paste.ubuntu.com/14867349/

Please re-open if you still have a problem.
Thanks,

[Xing Yang (xing-yang)]

Peter re-tested it and found that the original problem was fixed. He got a new error when deleting the temp volume during retyping:

2322740 2016-02-03 20:20:03.141 215234 ERROR oslo_messaging.rpc.dispatcher RPCVersionCapError: Requested message version, 1.36 is incompatible. It needs to be equal in major version and less than or equal in minor version as the specified version cap 1.30.

Will investigate more on the new error and report a new bug if needed.

[Xing Yang (xing-yang)]

Peter re-tested it with a fresh devstack setup and the problem went away.

[Mitsuhiro Tanino (mitsuhiro-tanino)]

Peter, Xyang,

Thank you four the confirmation. Now this bug can be closed as Duplicate.

[Michal Dulko (michal-dulko-f)]

Xing: Just FYI - the error Peter received was due to RPC version pinning stuff we've merged to enable rolling upgrades. In general it seems like in his deployment there were services in different versions and the older one wasn't able to accept newer requests.