Trying to get help for a command errors out with "__init__() got an unexpected keyword argument 'project_name'"

[Matt Riedemann]

Fresh ocata devstack created yesterday:

stack@filters:~$ pip show python-openstackclient
Name: python-openstackclient
Version: 3.4.1
Summary: OpenStack Command-line Client
Home-page: http://docs.openstack.org/developer/python-openstackclient
Author: OpenStack
Author-email: <email address hidden>
License: UNKNOWN
Location: /usr/local/lib/python2.7/dist-packages
Requires: oslo.i18n, python-cinderclient, osc-lib, keystoneauth1, six, python-glanceclient, openstacksdk, oslo.utils, cliff, python-novaclient, Babel, python-keystoneclient, pbr
stack@filters:~$

stack@filters:~$ openstack help quota set
__init__() got an unexpected keyword argument 'project_name'
stack@filters:~$

This time with --debug:

stack@filters:~$ openstack --debug help quota set
START with options: [u'--debug', u'help', u'quota', u'set']
options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://9.5.126.8:5000/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, old_profile=None, openid_scope='', os_baremetal_api_version='1.9', os_beta_command=False, os_compute_api_version='', os_dns_api_version='2', os_identity_api_version='3', os_image_api_version='', os_key_manager_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_volume_api_version='2', passcode='', password='***', profile=None, project_domain_id='default', project_domain_name='', project_id='', project_name='demo', protocol='', redirect_uri='', region_name='RegionOne', timing=False, token='***', trust_id='', url='', user_domain_id='default', user_domain_name='', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://9.5.126.8:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', u'status': u'active', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': '1.9', 'auth': {'project_name': 'demo', 'user_domain_id': 'default', 'project_domain_id': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, u'key_manager_api_version': '1', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': '2', 'username': 'admin', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
Deferring keystone exception: argument of type 'OSC_Config' is not iterable
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://9.5.126.8:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', u'status': u'active', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': '1.9', 'auth': {'project_name': 'demo', 'auth_url': 'http://9.5.126.8:5000/v3', 'user_domain_id': 'default', 'project_domain_id': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, u'key_manager_api_version': '1', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': '2', 'username': 'admin', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
compute API version 2, cmd group openstack.compute.v2
network API version 2, cmd group openstack.network.v2
image API version 2, cmd group openstack.image.v2
volume API version 2, cmd group openstack.volume.v2
identity API version 3, cmd group openstack.identity.v3
object_store API version 1, cmd group openstack.object_store.v1
neutronclient API version 2, cmd group openstack.neutronclient.v2
baremetal API version 1.9, cmd group openstack.baremetal.v1
orchestration API version 1, cmd group openstack.orchestration.v1
dns API version 2, cmd group openstack.dns.v2
key_manager API version 1, cmd group openstack.key_manager.v1
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://9.5.126.8:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', u'status': u'active', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': '1.9', 'auth': {'project_name': 'demo', 'user_domain_id': 'default', 'project_domain_id': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, u'key_manager_api_version': '1', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': '2', 'username': 'admin', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
Auth plugin token_endpoint selected
auth_config_hook(): {'auth_type': 'token_endpoint', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'timing': False, 'auth_url': 'http://9.5.126.8:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': '1.9', 'auth': {'url': 'x', 'token': '***', 'project_name': 'demo', 'user_domain_id': 'default', 'project_domain_id': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': '1', 'cacert': None, 'password': '***', u'key_manager_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': '2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
Deferring keystone exception: __init__() got an unexpected keyword argument 'project_name'
command: help -> cliff.help.HelpCommand
__init__() got an unexpected keyword argument 'project_name'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 90, in run
    return self.take_action(parsed_args) or 0
  File "/usr/local/lib/python2.7/dist-packages/cliff/help.py", line 91, in take_action
    cmd_parser = cmd.get_parser(full_name)
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/quota.py", line 109, in get_parser
    for k, v in self._build_options_list():
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/quota.py", line 86, in _build_options_list
    if self.app.client_manager.is_network_endpoint_enabled():
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/clientmanager.py", line 90, in is_network_endpoint_enabled
    return self.is_service_available('network') is not False
  File "/usr/local/lib/python2.7/dist-packages/osc_lib/clientmanager.py", line 203, in is_service_available
    if self.auth_ref:
  File "/usr/local/lib/python2.7/dist-packages/osc_lib/clientmanager.py", line 194, in auth_ref
    self.setup_auth()
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/clientmanager.py", line 78, in setup_auth
    self._cli_options.config,
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/client_config.py", line 196, in load_auth_plugin
    auth_plugin = loader.load_from_options(**config['auth'])
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/base.py", line 164, in load_from_options
    return self.create_plugin(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/base.py", line 125, in create_plugin
    return self.plugin_class(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'project_name'
clean_up HelpCommand: __init__() got an unexpected keyword argument 'project_name'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 135, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 279, in run
    result = self.run_subcommand(remainder)
  File "/usr/local/lib/python2.7/dist-packages/osc_lib/shell.py", line 180, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/local/lib/python2.7/dist-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/local/lib/python2.7/dist-packages/cliff/command.py", line 90, in run
    return self.take_action(parsed_args) or 0
  File "/usr/local/lib/python2.7/dist-packages/cliff/help.py", line 91, in take_action
    cmd_parser = cmd.get_parser(full_name)
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/quota.py", line 109, in get_parser
    for k, v in self._build_options_list():
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/quota.py", line 86, in _build_options_list
    if self.app.client_manager.is_network_endpoint_enabled():
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/clientmanager.py", line 90, in is_network_endpoint_enabled
    return self.is_service_available('network') is not False
  File "/usr/local/lib/python2.7/dist-packages/osc_lib/clientmanager.py", line 203, in is_service_available
    if self.auth_ref:
  File "/usr/local/lib/python2.7/dist-packages/osc_lib/clientmanager.py", line 194, in auth_ref
    self.setup_auth()
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/clientmanager.py", line 78, in setup_auth
    self._cli_options.config,
  File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/client_config.py", line 196, in load_auth_plugin
    auth_plugin = loader.load_from_options(**config['auth'])
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/base.py", line 164, in load_from_options
    return self.create_plugin(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/base.py", line 125, in create_plugin
    return self.plugin_class(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'project_name'

END return value: 1
stack@filters:~$

[Matt Riedemann (mriedem)]

BTW, with this devstack I'm sourcing creds by doing:

source devstack/openrc admin

[Matt Riedemann (mriedem)]

stack@filters:~$ printenv | grep OS_
OS_PROJECT_DOMAIN_ID=default
OS_REGION_NAME=RegionOne
OS_USER_DOMAIN_ID=default
OS_PROJECT_NAME=demo
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=****
OS_AUTH_URL=http://9.5.126.8:5000/v3
OS_USERNAME=admin
OS_TENANT_NAME=demo
OS_VOLUME_API_VERSION=2
OS_NO_CACHE=1

[Steve Martinelli (stevemar)]

Looks like bug https://bugs.launchpad.net/python-openstackclient/+bug/1652027 also reported this behaviour.

Dr. Jens:

When OS_PROJECT_NAME/OS_TENANT_NAME is set from sourcing openrc, displaying this help text fails:

$ openstack help security group rule create
__init__() got an unexpected keyword argument 'project_name'
$ openstack --version
openstack 3.6.0

It does work with no environment variables set, though.

[Steve Martinelli (stevemar)]

Looks like this only happens with networking commands. With the positional help and --help.

[Steve Martinelli (stevemar)]

Oh this is an interesting line:

"Deferring keystone exception: __init__() got an unexpected keyword argument 'project_name'"

So it does happen with non-networking commands, but the exception is ignored.

stevemar@ubuntu:/opt/stack/python-openstackclient$ openstack image list --help --debug
START with options: [u'image', u'list', u'--help', u'--debug']
options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://172.16.240.203/identity/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, old_profile=None, openid_scope='', os_baremetal_api_version='1.9', os_beta_command=False, os_compute_api_version='', os_dns_api_version='2', os_identity_api_version='3', os_image_api_version='', os_key_manager_api_version='1', os_network_api_version='', os_object_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_volume_api_version='2', passcode='', password='***', profile=None, project_domain_id='default', project_domain_name='', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='RegionOne', timing=False, token='***', trust_id='', url='', user_domain_id='default', user_domain_name='', user_id='', username='admin', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', u'orchestration_api_version': '1', u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://172.16.240.203/identity/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', u'status': u'active', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': '1.9', 'auth': {'project_name': 'admin', 'user_domain_id': 'default', 'project_domain_id': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': None, 'timing': False, 'password': '***', 'cacert': None, u'key_manager_api_version': '1', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': '2', 'username': 'admin', 'cert': None, u'secgroup_source': u'neutron', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
Deferring keystone exception: argument of type 'OSC_Config' is not iterable
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version':...

[Rui Chen (kiwik-chenrui)]

I found not all networking commands broken, for example: openstack network set --help, it's ok, but openstack network list --help failed.

Exception will be raised from self.app.client_manager.is_network_endpoint_enabled() method, the method have to get service catalog by a whole authentication, the key point is here[1], auth_type and config["auth"] are mismatch, "username" and other auth info are passed into token_endpoint.Token to initial.

[1]: https://github.com/openstack/python-openstackclient/blob/master/openstackclient/common/client_config.py#L196-L196

[Rui Chen (kiwik-chenrui)]

1. interactive_mode == True
  1.1 network list --help ok
  1.2 help network list failed
2. interactive_mode == False
  2.1 openstack network list --help failed
  2.2 openstack help network list failed

[Rui Chen (kiwik-chenrui)]

I don't know why the patch link isn't appended in bug comments automatically, add it:

https://review.openstack.org/#/c/418221/

[Chris Tomas (ctomas)]

Hi

When I am using devstack (openstack version 3.4.1) the problem appears too with:
$ openstack help security group rule create
$ openstack security group rule create -h

Now, I'm installing OpenStack in my NUC, using the Netwon installation tutorial for ubuntu 16.04. My openstack version is 3.2.0 and at the moment only installed identity service:

$ openstack service list
+----------------------------------+----------+----------+
| ID | Name | Type |
+----------------------------------+----------+----------+
| f8983c2b4a444203a1614f99abaf097a | keystone | identity |
+----------------------------------+----------+----------+

When I run the networking commands the problem not appears, inclusive the security group commands work fine.

[Rui Chen (kiwik-chenrui)]

Hi Chris:

We discuss the bug in IRC meeting yesterday and have a solution(see details in meeting log), I'm trying to fix the bug today. By the way, you can use openstackclient in interactive mode, the deferred help is ok, see my comment #7.

[Chris Tomas (ctomas)]

Thanks for the answer Rui!

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rui Chen (<email address hidden>) on branch: master
Review: https://review.openstack.org/418220
Reason: We decide to pop fake token and try to auth again, the patch isn't necessary.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/418221
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=4d9da2c40ae02086258cfde852b297754d8085fa
Submitter: Jenkins
Branch: master

commit 4d9da2c40ae02086258cfde852b297754d8085fa
Author: Rui Chen <email address hidden>
Date: Tue Jan 10 15:13:47 2017 +0800

    Fix OSC networking commands help errors

    OSC networking commands need to authenticate to get
    service catalog, then decide to show nova-network or
    neutron command help message. Fake token and fake
    auth_type in prepare_to_run_command() casue os-cloud-config
    use AdminToken auth plugin, but pass all the auth information
    (include: username, password and so on) to it, that casue the
    class initialization error. Pop the fake token and url, then
    try to load auth plugin again to fix the issue.

    Change-Id: I8b140f0b0a60681fc2a35a013bb0c84ff8cb9589
    Closes-Bug: #1650026

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-openstackclient 3.8.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/429653

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/429653
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=e79eca55aae02206bd10e663b1540078424d9734
Submitter: Jenkins
Branch: stable/newton

commit e79eca55aae02206bd10e663b1540078424d9734
Author: Rui Chen <email address hidden>
Date: Tue Jan 10 15:13:47 2017 +0800

    Fix OSC networking commands help errors

    OSC networking commands need to authenticate to get
    service catalog, then decide to show nova-network or
    neutron command help message. Fake token and fake
    auth_type in prepare_to_run_command() casue os-cloud-config
    use AdminToken auth plugin, but pass all the auth information
    (include: username, password and so on) to it, that casue the
    class initialization error. Pop the fake token and url, then
    try to load auth plugin again to fix the issue.

    Change-Id: I8b140f0b0a60681fc2a35a013bb0c84ff8cb9589
    Closes-Bug: #1650026
    (cherry picked from commit 4d9da2c40ae02086258cfde852b297754d8085fa)