Comment 1 for bug 1492177

This is not a bug, this is feature:) Different deployments have different default roles. For example, it can be "Member" or "_member_".

If you need to specify role to users, created by rally, you can use RoleContext - https://github.com/openstack/rally/blob/master/rally/plugins/openstack/context/keystone/roles.py#L28

Your scenario should looks like - http://paste.openstack.org/show/445230/