[SRU] openstack command raises exception referencing gi.repository and gnome bug 709183

[Scott Moser]

$ openstack
Exception raised: When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject". See: https://bugzilla.gnome.org/show_bug.cgi?id=709183

$ sstack openstack --debug
START with options: [u'--debug']
options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://10.245.161.156:5000/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, openid_scope='', os_beta_command=False, os_compute_api_version='', os_identity_api_version='3', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='', passcode='', password='***', project_domain_id='', project_domain_name='user', project_id='6286a8c4f88046b3bdf25b26ff54f1f7', project_name='smoser', protocol='', redirect_uri='', region_name='serverstack', service_provider_endpoint='', service_provider_entity_id='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='user', user_id='', username='smoser', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://10.245.161.156:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'smoser', u'container_infra_api_version': u'1', 'verbose_level': 3, 'region_name': 'serverstack', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_id': '6286a8c4f88046b3bdf25b26ff54f1f7', 'user_domain_name': 'user', 'project_name': 'smoser', 'project_domain_name': 'user'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'container_infra_api_version': u'1', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'application_catalog_api_version': u'1', u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://10.245.161.156:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'smoser', u'container_infra_api_version': u'1', 'verbose_level': 3, 'region_name': 'serverstack', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_id': '6286a8c4f88046b3bdf25b26ff54f1f7', 'user_domain_name': 'user', 'project_name': 'smoser', 'project_domain_name': 'user'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
compute API version 2, cmd group openstack.compute.v2
network API version 2, cmd group openstack.network.v2
image API version 2, cmd group openstack.image.v2
volume API version 2, cmd group openstack.volume.v2
identity API version 3, cmd group openstack.identity.v3
object_store API version 1, cmd group openstack.object_store.v1
neutronclient API version 2, cmd group openstack.neutronclient.v2
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://10.245.161.156:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'smoser', u'container_infra_api_version': u'1', 'verbose_level': 3, 'region_name': 'serverstack', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_id': '6286a8c4f88046b3bdf25b26ff54f1f7', 'user_domain_name': 'user', 'project_name': 'smoser', 'project_domain_name': 'user'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 277, in run
    result = self.interact()
  File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 177, in interact
    ret_value = super(OpenStackShell, self).interact()
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 316, in interact
    from .interactive import InteractiveApp
  File "/usr/lib/python2.7/dist-packages/cliff/interactive.py", line 20, in <module>
    import cmd2
  File "/usr/lib/python2.7/dist-packages/cmd2.py", line 312, in <module>
    _ = pyperclip.paste()
  File "/usr/lib/python2.7/dist-packages/pyperclip/clipboards.py", line 37, in paste_gtk
    clipboardContents = gtk.Clipboard().wait_for_text()
  File "/usr/lib/python2.7/dist-packages/gi/__init__.py", line 64, in __getattr__
    raise AttributeError(_static_binding_error)
AttributeError: When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject". See: https://bugzilla.gnome.org/show_bug.cgi?id=709183

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: python-openstackclient 3.12.0-0ubuntu2
ProcVersionSignature: Ubuntu 4.13.0-12.13-generic 4.13.3
Uname: Linux 4.13.0-12-generic x86_64
NonfreeKernelModules: zfs zunicode zavl zcommon znvpair
ApportVersion: 2.20.7-0ubuntu2
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Tue Oct 10 10:58:55 2017
EcryptfsInUse: Yes
InstallationDate: Installed on 2015-07-23 (810 days ago)
InstallationMedia: Ubuntu 15.10 "Wily Werewolf" - Alpha amd64 (20150722.1)
PackageArchitecture: all
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: python-openstackclient
UpgradeStatus: No upgrade log present (probably fresh install)

[Scott Moser (smoser)]

note that 'openstack server list' does work, its just when trying to start the interactive shell it fails.

[Corey Bryant (corey.bryant)]

This looks to be very similar to: https://github.com/mps-youtube/mps-youtube/issues/461

[Corey Bryant (corey.bryant)]

Here's how the code flows to python-keyring where the conflicting code appears to be:

openstackclient/shell.py
  from openstackclient.common import clientmanager

openstackclient/common/clientmanager.py
   # Get list of base plugin modules
   PLUGIN_MODULES = get_plugin_modules(
       'openstack.cli.base',
   )

   ^ loops through and imports the following modules from setup.cfg

   openstack.cli.base =
     compute = openstackclient.compute.client
     identity = openstackclient.identity.client
     image = openstackclient.image.client
     network = openstackclient.network.client
     object_store = openstackclient.object.client
     volume = openstackclient.volume.client

openstackclient/identity/client.py:
  from keystoneclient.v2_0 import client as identity_client_v2

keystoneclient/v2_0/client.py:
  from keystoneclient import httpclient

keystoneclient/httpclient.py
  import keyring

keyring/__init__.py:
  logger = logging.getLogger('keyring')
  (move this line ^ to bottom of file and issue goes away)

[Corey Bryant (corey.bryant)]

I'm not sure that the logging.getLogger('keyring') is necessary, but it also doesn't seem to be the root cause of this.

This bug has the reasoning behind why the AttributeError exception is thrown: https://bugzilla.gnome.org/show_bug.cgi?id=709183

[Corey Bryant (corey.bryant)]

This can be reproduced with the following python script:

#!/usr/bin/env python

import keyring
import pyperclip

pyperclip.paste()

[Corey Bryant (corey.bryant)]

Narrowing down a bit more, this can also be reproduced with the following python script:

#!/usr/bin/env python

import gtk
import gi

The 2 lines of code that appear to be conflicting are:
1) /usr/lib/python2.7/dist-packages/keyrings/alt/Gnome.py
   import gi
2) /usr/lib/python2.7/dist-packages/pyperclip/clipboards.py
   import gtk

And if I understand correctly from https://bugzilla.gnome.org/show_bug.cgi?id=709183, this is importing gtk+2 modules via 'import gtk' as well as gtk+3 modules via 'import gi'.

From http://www.pygtk.org/: Note: New users are encouraged to use GTK+3 through the PyGObject bindings instead of using PyGTK with GTK+2.

So it seems to me that pyperclip in artful needs to move to GTK+3.

[Corey Bryant (corey.bryant)]

Edit to previous comment:

And if I understand correctly from https://bugzilla.gnome.org/show_bug.cgi?id=709183, this is importing gtk+2 modules via 'import gtk' as well as gtk+3 modules via 'import gi' **which causes module conflicts**.

[Corey Bryant (corey.bryant)]

Another edit:

And if I understand correctly from https://bugzilla.gnome.org/show_bug.cgi?id=709183, this is importing gtk+2 modules via 'import gtk' as well as gtk+3 modules via 'import gi' which causes module conflicts **(according to the same bug)**.

[Corey Bryant (corey.bryant)]

According to https://bugzilla.gnome.org/show_bug.cgi?id=709183, gtk+2 and gtk+3 are mutually exclusive.

I've opened https://github.com/asweigart/pyperclip/issues/106 with the upstream pyperclip project.

[Corey Bryant (corey.bryant)]

Checking to see if we can work around this via a change to keyring: https://github.com/jaraco/keyring/issues/291

[Corey Bryant (corey.bryant)]

The only reason I've marked python-openstackclient as low is because the bug is not in that code. The fix is required in python-pyperclient and cmd2.

I've not received any action on the upstream pyperclip bug, so I submitted patches this morning to move pyperclip and cmd2 to GTL+ version 3:

https://github.com/asweigart/pyperclip/pull/111
https://github.com/python-cmd2/cmd2/pull/242

I'd like to get a quick upstream review on those before I backport to Ubuntu.

[Corey Bryant (corey.bryant)]

s/GTL/GDK

[Corey Bryant (corey.bryant)]

sigh.. s/GDK/GTK

[Corey Bryant (corey.bryant)]

It seems this issue has gone away in artful but still exists in bionic.

[Corey Bryant (corey.bryant)]

I've uploaded patched cmd2 and python-pyperclip packages to Bionic to fix this issue. If you're still seeing this on any other releases please let me know.

[Launchpad Janitor (janitor)]

This bug was fixed in the package python-pyperclip - 1.5.32-1ubuntu1

---------------
python-pyperclip (1.5.32-1ubuntu1) bionic; urgency=medium

  * d/p/move-to-gtk3.patch: Use GTK+3 instead of GTK+2 (LP: #1722553).

 -- Corey Bryant <email address hidden> Tue, 19 Dec 2017 16:25:22 -0500

[Corey Bryant (corey.bryant)]

I just had a report that this is still occurring in Artful. While I can't recreate on a new kvm instance, I want to open this back up for Artful and possibly provide a fix or get to the bottom of why I'm not seeing it.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Conrad Rockenhaus (1-conrad)]

I can confirm that this issue is still occurring in Artful. Use of the client from the command line works just fine, but any attempt to use it as an interactive shell fails with this error.

[Scott Moser (smoser)]

Conrad,
The bug is definitely still present in Artful. Fixing it in Artful (a stable release) is what is known as a Stable Release Update [1]. I can verify I do not see the problem with just 'openstack' on bionic.

$ dpkg -S $(readlink -f `which openstack`)
python3-openstackclient: /usr/bin/python3-openstack
$ dpkg-query --show python3-openstackclient
python3-openstackclient 3.14.0-0ubuntu1

$ openstack
(openstack)

So that seems fixed in bionic (and will be fixed in 18.04).

Note that just in this testing, I found bug 1751822.

--
[1] https://wiki.ubuntu.com/StableReleaseUpdates

[Corey Bryant (corey.bryant)]

This error doesn't surface on the Pike cloud-archive, but I've added it as the Artful version will be backported to Pike and will need regression testing.

[Corey Bryant (corey.bryant)]

I've uploaded a patched version of python-pyperclip to the Artful queue where it is awaiting SRU team review: https://launchpad.net/ubuntu/artful/+queue?queue_state=1&queue_text=

[Corey Bryant (corey.bryant)]

[Impact]
The openstack CLI interactive shell is unusable. More details in bug description above.

[Test Case]
Install python-openstackclient and attempt to enter the interactive shell:

$ openstack
Exception raised: When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject". See: https://bugzilla.gnome.org/show_bug.cgi?id=709183

[Regression Potential]
A similar fix landed in bionic on Jan 2nd and hasn't had any issues so far.

[Discussion]

[Brian Murray (brian-murray)]

Hello Scott, or anyone else affected,

Accepted python-pyperclip into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-pyperclip/1.5.27-3ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Corey Bryant (corey.bryant)]

It turns out we don't carry pyperclip or cmd2 in the pike cloud archive so I'm dropping that target from this bug.

[Annie Melen (anniemelen)]

I faced the same exception when tried to upgrade python packages accordingly to stable/queens upper_constraints: https://github.com/openstack/requirements/blob/stable/queens/upper-constraints.txt

Steps to reproduce
-----------------------
The following link shows steps to reproduce this issue
https://paste.ubuntu.com/p/NxcbSy5gsk/

Current workaround
-----------------------
Downgrade cmd2 packages to 0.6.8 version

My environment
-----------------------
Ubuntu 16.04.4 LTS (4.4.0-124-generic)
Openstack Queens
python-openstackclient 3.14.0
cmd2 0.7.9
pyperclip 1.6.0

[Corey Bryant (corey.bryant)]

Hi Annie,

It looks like you may have a mixture of pip-installed packages and Ubuntu packages which isn't supported. If that's the case I'd suggest testing in 2 different ways:

1) pip installed packages only, in a virtual env
2) ubuntu packages only

Thanks,
Corey

[Annie Melen (anniemelen)]

Hi, Corey!

What I've got after testing the first case:
a) At start, when I'm installing python-openstackclient only, openstack interactive mode is succeeded.
b) When I'm trying to install 'keystone' packages (with -c upper_constraints, of course), interactive mode is failed.
https://paste.ubuntu.com/p/Fb6CKThqwF/
c) Then I'm downgrading 'cmd2' packages to v.0.6.8, everything is fine.

Obviously, something is broken here...

Regards,
Annie

[Corey Bryant (corey.bryant)]

Hi Annie,

If you're hitting this issue when using pip to install python-openstackclient/cmd2 then it is likely an upstream issue. If that's the case can you start by opening an issue here? This is where upstream tracks their issues: https://storyboard.openstack.org/#!/project/975

If you're hitting the issue with apt (and only apt -- ie. no pip anywhere) it is likely a packaging issue.

If you're mixing use of apt and pip to install python packages then that's not supported.

Thanks,
Corey

[Corey Bryant (corey.bryant)]

To be clear for the 2nd paragraph above that refers to Ubuntu packaging, and we would track that here in Launchpad.

[Corey Bryant (corey.bryant)]

Finally, this *should* be fixed in Ubuntu packages as we're carrying patches to the upstream code in our packages. As for upstream, I submitted patches to pyperclip and cmd2 a while back. The pyperclip pull request unfortunately ended up being ignored and I'm not sure why [1].

[1] https://github.com/asweigart/pyperclip/issues/106, https://github.com/asweigart/pyperclip/pull/111