[SRU] openstack command raises exception referencing gi.repository and gnome bug 709183

Bug #1722553 reported by Scott Moser
42
This bug affects 7 people
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Fix Released
Undecided
Unassigned
cmd2 (Ubuntu)
Fix Released
High
Unassigned
python-openstackclient (Ubuntu)
Invalid
Low
Unassigned
python-pyperclip (Ubuntu)
Fix Released
High
Unassigned
Artful
Won't Fix
High
Unassigned

Bug Description

$ openstack
Exception raised: When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject". See: https://bugzilla.gnome.org/show_bug.cgi?id=709183

$ sstack openstack --debug
START with options: [u'--debug']
options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://10.245.161.156:5000/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, openid_scope='', os_beta_command=False, os_compute_api_version='', os_identity_api_version='3', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='', passcode='', password='***', project_domain_id='', project_domain_name='user', project_id='6286a8c4f88046b3bdf25b26ff54f1f7', project_name='smoser', protocol='', redirect_uri='', region_name='serverstack', service_provider_endpoint='', service_provider_entity_id='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='user', user_id='', username='smoser', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://10.245.161.156:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'smoser', u'container_infra_api_version': u'1', 'verbose_level': 3, 'region_name': 'serverstack', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_id': '6286a8c4f88046b3bdf25b26ff54f1f7', 'user_domain_name': 'user', 'project_name': 'smoser', 'project_domain_name': 'user'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'container_infra_api_version': u'1', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'application_catalog_api_version': u'1', u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://10.245.161.156:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'smoser', u'container_infra_api_version': u'1', 'verbose_level': 3, 'region_name': 'serverstack', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_id': '6286a8c4f88046b3bdf25b26ff54f1f7', 'user_domain_name': 'user', 'project_name': 'smoser', 'project_domain_name': 'user'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
compute API version 2, cmd group openstack.compute.v2
network API version 2, cmd group openstack.network.v2
image API version 2, cmd group openstack.image.v2
volume API version 2, cmd group openstack.volume.v2
identity API version 3, cmd group openstack.identity.v3
object_store API version 1, cmd group openstack.object_store.v1
neutronclient API version 2, cmd group openstack.neutronclient.v2
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://10.245.161.156:5000/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', 'username': 'smoser', u'container_infra_api_version': u'1', 'verbose_level': 3, 'region_name': 'serverstack', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_id': '6286a8c4f88046b3bdf25b26ff54f1f7', 'user_domain_name': 'user', 'project_name': 'smoser', 'project_domain_name': 'user'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_manager_api_version': u'v1', u'workflow_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'debug': True, u'interface': None, u'disable_vendor_agent': {}}
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 277, in run
    result = self.interact()
  File "/usr/lib/python2.7/dist-packages/osc_lib/shell.py", line 177, in interact
    ret_value = super(OpenStackShell, self).interact()
  File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 316, in interact
    from .interactive import InteractiveApp
  File "/usr/lib/python2.7/dist-packages/cliff/interactive.py", line 20, in <module>
    import cmd2
  File "/usr/lib/python2.7/dist-packages/cmd2.py", line 312, in <module>
    _ = pyperclip.paste()
  File "/usr/lib/python2.7/dist-packages/pyperclip/clipboards.py", line 37, in paste_gtk
    clipboardContents = gtk.Clipboard().wait_for_text()
  File "/usr/lib/python2.7/dist-packages/gi/__init__.py", line 64, in __getattr__
    raise AttributeError(_static_binding_error)
AttributeError: When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject". See: https://bugzilla.gnome.org/show_bug.cgi?id=709183

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: python-openstackclient 3.12.0-0ubuntu2
ProcVersionSignature: Ubuntu 4.13.0-12.13-generic 4.13.3
Uname: Linux 4.13.0-12-generic x86_64
NonfreeKernelModules: zfs zunicode zavl zcommon znvpair
ApportVersion: 2.20.7-0ubuntu2
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Tue Oct 10 10:58:55 2017
EcryptfsInUse: Yes
InstallationDate: Installed on 2015-07-23 (810 days ago)
InstallationMedia: Ubuntu 15.10 "Wily Werewolf" - Alpha amd64 (20150722.1)
PackageArchitecture: all
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: python-openstackclient
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Scott Moser (smoser) wrote :
Changed in python-openstackclient (Ubuntu):
status: New → Confirmed
Revision history for this message
Scott Moser (smoser) wrote :

note that 'openstack server list' does work, its just when trying to start the interactive shell it fails.

Revision history for this message
Corey Bryant (corey.bryant) wrote :
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Here's how the code flows to python-keyring where the conflicting code appears to be:

openstackclient/shell.py
  from openstackclient.common import clientmanager

openstackclient/common/clientmanager.py
   # Get list of base plugin modules
   PLUGIN_MODULES = get_plugin_modules(
       'openstack.cli.base',
   )

   ^ loops through and imports the following modules from setup.cfg

   openstack.cli.base =
     compute = openstackclient.compute.client
     identity = openstackclient.identity.client
     image = openstackclient.image.client
     network = openstackclient.network.client
     object_store = openstackclient.object.client
     volume = openstackclient.volume.client

openstackclient/identity/client.py:
  from keystoneclient.v2_0 import client as identity_client_v2

keystoneclient/v2_0/client.py:
  from keystoneclient import httpclient

keystoneclient/httpclient.py
  import keyring

keyring/__init__.py:
  logger = logging.getLogger('keyring')
  (move this line ^ to bottom of file and issue goes away)

Revision history for this message
Corey Bryant (corey.bryant) wrote :

I'm not sure that the logging.getLogger('keyring') is necessary, but it also doesn't seem to be the root cause of this.

This bug has the reasoning behind why the AttributeError exception is thrown: https://bugzilla.gnome.org/show_bug.cgi?id=709183

Revision history for this message
Corey Bryant (corey.bryant) wrote :

This can be reproduced with the following python script:

#!/usr/bin/env python

import keyring
import pyperclip

pyperclip.paste()

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Narrowing down a bit more, this can also be reproduced with the following python script:

#!/usr/bin/env python

import gtk
import gi

The 2 lines of code that appear to be conflicting are:
1) /usr/lib/python2.7/dist-packages/keyrings/alt/Gnome.py
   import gi
2) /usr/lib/python2.7/dist-packages/pyperclip/clipboards.py
   import gtk

And if I understand correctly from https://bugzilla.gnome.org/show_bug.cgi?id=709183, this is importing gtk+2 modules via 'import gtk' as well as gtk+3 modules via 'import gi'.

From http://www.pygtk.org/: Note: New users are encouraged to use GTK+3 through the PyGObject bindings instead of using PyGTK with GTK+2.

So it seems to me that pyperclip in artful needs to move to GTK+3.

Changed in python-pyperclip (Ubuntu):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Edit to previous comment:

And if I understand correctly from https://bugzilla.gnome.org/show_bug.cgi?id=709183, this is importing gtk+2 modules via 'import gtk' as well as gtk+3 modules via 'import gi' **which causes module conflicts**.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Another edit:

And if I understand correctly from https://bugzilla.gnome.org/show_bug.cgi?id=709183, this is importing gtk+2 modules via 'import gtk' as well as gtk+3 modules via 'import gi' which causes module conflicts **(according to the same bug)**.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

According to https://bugzilla.gnome.org/show_bug.cgi?id=709183, gtk+2 and gtk+3 are mutually exclusive.

I've opened https://github.com/asweigart/pyperclip/issues/106 with the upstream pyperclip project.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Checking to see if we can work around this via a change to keyring: https://github.com/jaraco/keyring/issues/291

Revision history for this message
Corey Bryant (corey.bryant) wrote :

The only reason I've marked python-openstackclient as low is because the bug is not in that code. The fix is required in python-pyperclient and cmd2.

I've not received any action on the upstream pyperclip bug, so I submitted patches this morning to move pyperclip and cmd2 to GTL+ version 3:

https://github.com/asweigart/pyperclip/pull/111
https://github.com/python-cmd2/cmd2/pull/242

I'd like to get a quick upstream review on those before I backport to Ubuntu.

Changed in cmd2 (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in python-openstackclient (Ubuntu):
importance: Undecided → Low
Revision history for this message
Corey Bryant (corey.bryant) wrote :

s/GTL/GDK

Revision history for this message
Corey Bryant (corey.bryant) wrote :

sigh.. s/GDK/GTK

Revision history for this message
Corey Bryant (corey.bryant) wrote :

It seems this issue has gone away in artful but still exists in bionic.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

I've uploaded patched cmd2 and python-pyperclip packages to Bionic to fix this issue. If you're still seeing this on any other releases please let me know.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-pyperclip - 1.5.32-1ubuntu1

---------------
python-pyperclip (1.5.32-1ubuntu1) bionic; urgency=medium

  * d/p/move-to-gtk3.patch: Use GTK+3 instead of GTK+2 (LP: #1722553).

 -- Corey Bryant <email address hidden> Tue, 19 Dec 2017 16:25:22 -0500

Changed in python-pyperclip (Ubuntu):
status: Triaged → Fix Released
Changed in cmd2 (Ubuntu):
status: Triaged → Fix Released
Changed in python-openstackclient (Ubuntu):
status: Confirmed → Invalid
Revision history for this message
Corey Bryant (corey.bryant) wrote :

I just had a report that this is still occurring in Artful. While I can't recreate on a new kvm instance, I want to open this back up for Artful and possibly provide a fix or get to the bottom of why I'm not seeing it.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in cmd2 (Ubuntu Artful):
status: New → Confirmed
Changed in python-openstackclient (Ubuntu Artful):
status: New → Confirmed
Changed in python-pyperclip (Ubuntu Artful):
status: New → Confirmed
Revision history for this message
Conrad Rockenhaus (conradrock) wrote :

I can confirm that this issue is still occurring in Artful. Use of the client from the command line works just fine, but any attempt to use it as an interactive shell fails with this error.

Revision history for this message
Scott Moser (smoser) wrote :

Conrad,
The bug is definitely still present in Artful. Fixing it in Artful (a stable release) is what is known as a Stable Release Update [1]. I can verify I do not see the problem with just 'openstack' on bionic.

$ dpkg -S $(readlink -f `which openstack`)
python3-openstackclient: /usr/bin/python3-openstack
$ dpkg-query --show python3-openstackclient
python3-openstackclient 3.14.0-0ubuntu1

$ openstack
(openstack)

So that seems fixed in bionic (and will be fixed in 18.04).

Note that just in this testing, I found bug 1751822.

--
[1] https://wiki.ubuntu.com/StableReleaseUpdates

no longer affects: python-openstackclient (Ubuntu Artful)
no longer affects: cmd2 (Ubuntu Artful)
no longer affects: cloud-archive
Revision history for this message
Corey Bryant (corey.bryant) wrote :

This error doesn't surface on the Pike cloud-archive, but I've added it as the Artful version will be backported to Pike and will need regression testing.

Changed in cloud-archive:
status: New → Fix Released
Revision history for this message
Corey Bryant (corey.bryant) wrote :

I've uploaded a patched version of python-pyperclip to the Artful queue where it is awaiting SRU team review: https://launchpad.net/ubuntu/artful/+queue?queue_state=1&queue_text=

Changed in python-pyperclip (Ubuntu Artful):
status: Confirmed → Triaged
importance: Undecided → High
Revision history for this message
Corey Bryant (corey.bryant) wrote :

[Impact]
The openstack CLI interactive shell is unusable. More details in bug description above.

[Test Case]
Install python-openstackclient and attempt to enter the interactive shell:

$ openstack
Exception raised: When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject". See: https://bugzilla.gnome.org/show_bug.cgi?id=709183

[Regression Potential]
A similar fix landed in bionic on Jan 2nd and hasn't had any issues so far.

[Discussion]

summary: - openstack command raises exception referencing gi.repository and gnome
- bug 709183
+ [SRU] openstack command raises exception referencing gi.repository and
+ gnome bug 709183
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Scott, or anyone else affected,

Accepted python-pyperclip into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-pyperclip/1.5.27-3ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-pyperclip (Ubuntu Artful):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-artful
Revision history for this message
Corey Bryant (corey.bryant) wrote :

It turns out we don't carry pyperclip or cmd2 in the pike cloud archive so I'm dropping that target from this bug.

no longer affects: cloud-archive/pike
no longer affects: cloud-archive/pike
Revision history for this message
Annie Melen (anniemelen) wrote :

I faced the same exception when tried to upgrade python packages accordingly to stable/queens upper_constraints: https://github.com/openstack/requirements/blob/stable/queens/upper-constraints.txt

Steps to reproduce
-----------------------
The following link shows steps to reproduce this issue
https://paste.ubuntu.com/p/NxcbSy5gsk/

Current workaround
-----------------------
Downgrade cmd2 packages to 0.6.8 version

My environment
-----------------------
Ubuntu 16.04.4 LTS (4.4.0-124-generic)
Openstack Queens
python-openstackclient 3.14.0
cmd2 0.7.9
pyperclip 1.6.0

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hi Annie,

It looks like you may have a mixture of pip-installed packages and Ubuntu packages which isn't supported. If that's the case I'd suggest testing in 2 different ways:

1) pip installed packages only, in a virtual env
2) ubuntu packages only

Thanks,
Corey

Revision history for this message
Annie Melen (anniemelen) wrote :

Hi, Corey!

What I've got after testing the first case:
a) At start, when I'm installing python-openstackclient only, openstack interactive mode is succeeded.
b) When I'm trying to install 'keystone' packages (with -c upper_constraints, of course), interactive mode is failed.
https://paste.ubuntu.com/p/Fb6CKThqwF/
c) Then I'm downgrading 'cmd2' packages to v.0.6.8, everything is fine.

Obviously, something is broken here...

Regards,
Annie

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hi Annie,

If you're hitting this issue when using pip to install python-openstackclient/cmd2 then it is likely an upstream issue. If that's the case can you start by opening an issue here? This is where upstream tracks their issues: https://storyboard.openstack.org/#!/project/975

If you're hitting the issue with apt (and only apt -- ie. no pip anywhere) it is likely a packaging issue.

If you're mixing use of apt and pip to install python packages then that's not supported.

Thanks,
Corey

Revision history for this message
Corey Bryant (corey.bryant) wrote :

To be clear for the 2nd paragraph above that refers to Ubuntu packaging, and we would track that here in Launchpad.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Finally, this *should* be fixed in Ubuntu packages as we're carrying patches to the upstream code in our packages. As for upstream, I submitted patches to pyperclip and cmd2 a while back. The pyperclip pull request unfortunately ended up being ignored and I'm not sure why [1].

[1] https://github.com/asweigart/pyperclip/issues/106, https://github.com/asweigart/pyperclip/pull/111

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Artful is EOL so I'm marking it as Won't Fix at this point, though the fix is/was available in artful-proposed.

Changed in python-pyperclip (Ubuntu):
status: Fix Released → Won't Fix
status: Won't Fix → Fix Released
Changed in python-pyperclip (Ubuntu Artful):
status: Fix Committed → Won't Fix
Revision history for this message
admgsic (j-3dmin-q) wrote :

I have faced the same problem while I was installing Openstack Train on Ubuntu Server 18.04.3 LTS.

a) Firstly, when I have installed python-openstackclient, keystone and glance, openstack interactive mode is succeeded.

b) When I have installes 'placement' packages interactive mode is failed.

I'm using:
 python-openstackclient (3.14.2)
 cmd2 (0.8.9)
 pyperclip (1.7.0)

The solution was like https://bugs.launchpad.net/cloud-archive/+bug/1722553/comments/31 downgrade cmd2 packages to 0.6.8 version

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.