Unable to autolaunch a dbus-daemon without a $DISPLAY for X11

[Lorin Hochstein]

Tried to install Openstack using devstack on saucy (server edition) inside of vagrant, and got the following error:

++ keystone tenant-create --name admin
Traceback (most recent call last):
  File "/usr/local/bin/keystone", line 6, in <module>
    from keystoneclient.shell import main
  File "/opt/stack/python-keystoneclient/keystoneclient/shell.py", line 36, in <module>
    from keystoneclient.contrib.bootstrap import shell as shell_bootstrap
  File "/opt/stack/python-keystoneclient/keystoneclient/contrib/bootstrap/shell.py", line 16, in <module>
    from keystoneclient.v2_0 import client
  File "/opt/stack/python-keystoneclient/keystoneclient/v2_0/__init__.py", line 2, in <module>
    from keystoneclient.v2_0.client import Client
  File "/opt/stack/python-keystoneclient/keystoneclient/v2_0/client.py", line 18, in <module>
    from keystoneclient import httpclient
  File "/opt/stack/python-keystoneclient/keystoneclient/httpclient.py", line 32, in <module>
    import keyring
  File "/usr/local/lib/python2.7/dist-packages/keyring/__init__.py", line 9, in <module>
    from core import (set_keyring, get_keyring, set_password, get_password,
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 166, in <module>
    init_backend()
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 65, in init_backend
    keyrings.sort(key = lambda x: -x.supported())
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 65, in <lambda>
    keyrings.sort(key = lambda x: -x.supported())
  File "/usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py", line 18, in supported
    bus = secretstorage.dbus_init()
  File "/usr/lib/python2.7/dist-packages/secretstorage/__init__.py", line 42, in dbus_init
    return dbus.SessionBus()
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 211, in __new__
    mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 100, in __new__
    bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/bus.py", line 122, in __new__
    bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NotSupported: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11

Same error if I try to invoke a keystone command from the shell.

[Dolph Mathews (dolph)]

As a workaround, I'd recommend uninstalling keyring if you don't need it.

[Lorin Hochstein (lorinh)]

I'm trying to figure out where in the process it's getting added so I can remove it before the failures happen during hte devestack process.

I see the python-keystoneclient has keyring as requirement in test-requirements.txt. Does DevStack install packages from test-requirements.txt?

[Manuel Stein (manuel.stein)]

Keyring is imported by tempest and python-openstackclient
./tempest/requirements.txt:keyring>=1.6.1
./python-openstackclient/requirements.txt:keyring>=1.6.1,<2.0
Though python-keystoneclient optionally imports the keyring backend, it hits admin account creation in devstack first.
It's a configuration issue of python-keyring, an invocation of set_keyring would help, but wouldn't respect user preferences.

It's a matter of the SecretService backend. I got it running by modifying
/usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py
The ClassProperty "priority" tries to initialize dbus before checking for the display variable. Just pull the if clause upfront and you have a working RuntimeException that keeps keyring from trying to use it:

   def priority(cls):
        if not 'secretstorage' in globals():
            raise RuntimeError("SecretService required")
        if 'DISPLAY' not in os.environ:
            raise RuntimeError("SecretService cannot run without a DISPLAY "
                "environment variable")
        try:
            bus = secretstorage.dbus_init()
            secretstorage.Collection(bus)
        except secretstorage.exceptions.SecretServiceNotAvailableException:
            raise RuntimeError("Unable to get initialize SecretService")
        return 5

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Manuel Stein (manuel.stein)]

I filed a bug with python-keyring:
https://bitbucket.org/kang/python-keyring-lib/issue/122/error-in-secretservice-priority-cls
I assume it won't be fixed in v <2.0 as required by python-openstackclient.
Maybe recommend an example keyring configuration during installation of affected projects?

[Lorin Hochstein (lorinh)]

I worked around this in my devstack-vm repo <https://github.com/lorin/devstack-vm> by adding creating a file at ~/.local/share/python_keyring/keyringrc.cfg:

[backend]
default-keyring=keyring.backends.file.PlaintextKeyring

This is insecure (passwords are stored in plaintext), but for a devstack deployment it works just fine.

[Peter Feiner (pete5)]

I had the same issue. In my case, python-keyring was installed as a dependency of python-apport, which I needed for to report a bug to Canonical.

[Adam Gandelman (gandelman-a)]

Can someone please capture how this bug affects the Ubunty python-keystoneclient package and how to reproduce with whats in the archive (saucy or trusty)? The original trace appears to implicate non-Ubuntu stuff installed by devstack (pip installed python-keyring, devstack installed python-keystoneclient). This looks more like a python-keyring or devstack issue?

[Sina Sadeghi (sina-sa)]

Hi Adam,

I experienced this bug on upgrading an Ubuntu 12.04 server running Grizzly Horizon to Havana using the Cloud Archive. The server was installed less than 6 months ago from scratch and never had devstack on it. Running apt-get --purge remove python-keyring fixed the issue for me.

[Sina Sadeghi (sina-sa)]

(I definitely did not install anything on this box with pip, nor ever explicitly install python-keyring)

[Darryl Weaver (dweaver)]

I just hit this bug.

I am deploying Havana using Juju charms, so it is a precise deployment of Havana from the cloud archive.
The deployment was working correctly a few days ago, so it is a recent update that causes the issue.

The python-keyring package installed was 0.9.2-0ubuntu0.12.04.2.

I can confirm that running apt-get --purge remove python-keyring fixes the issue, but also removes the packages: apport python-apport python-launchpadlib.

So, it is definitely a bug that will effect cloud-archive users and is not specific to devstack.

[Darryl Weaver (dweaver)]

Error is seen in the apache log file when trying to access openstack-dashboard web interface.

[Joe Hakim Rahme (rahmu)]

I just hit this bug (vagrant + ubuntu saucy + first devstack install on this VM). Removing python-keyring didn't solve the problem. I reinstalled it and set its backend to plaintext files ([as suggested here][1]) and that workaround seems to work.

 [1]: https://bugs.launchpad.net/ubuntu/+source/python-keystoneclient/+bug/1242992/comments/6

[Terry Howe (thowe-g)]

This bug should of been fixed in keyring 3.3 which came after this commit:

https://bitbucket.org/kang/python-keyring-lib/diff/keyring/backends/SecretService.py?diff2=f299f5a0ea3d&at=default

[Mike Spreitzer (mike-spreitzer)]

I am still getting this. I have the most recent keyring:

$ pip show keyring
---
Name: keyring
Version: 3.4
Location: /usr/local/lib/python2.7/dist-packages
Requires:

This is a DevStack install of OpenStack onto an Ubuntu 13.10 machine. This shows the vintage of the python-keystoneclient (it was around Feb 12):

$ git branch -v
* master 5b3342d Merge "Fix debug curl commands for included data"

Here is a typescript showing the problem happening:

$ keystone help
Traceback (most recent call last):
  File "/usr/local/bin/keystone", line 6, in <module>
    from keystoneclient.shell import main
  File "/opt/stack/python-keystoneclient/keystoneclient/shell.py", line 37, in <module>
    from keystoneclient.contrib.bootstrap import shell as shell_bootstrap
  File "/opt/stack/python-keystoneclient/keystoneclient/contrib/bootstrap/shell.py", line 16, in <module>
    from keystoneclient.v2_0 import client
  File "/opt/stack/python-keystoneclient/keystoneclient/v2_0/__init__.py", line 2, in <module>
    from keystoneclient.v2_0.client import Client
  File "/opt/stack/python-keystoneclient/keystoneclient/v2_0/client.py", line 19, in <module>
    from keystoneclient import httpclient
  File "/opt/stack/python-keystoneclient/keystoneclient/httpclient.py", line 29, in <module>
    import keyring
  File "/usr/local/lib/python2.7/dist-packages/keyring/__init__.py", line 12, in <module>
    from .core import (set_keyring, get_keyring, set_password, get_password,
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 180, in <module>
    init_backend()
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 59, in init_backend
    set_keyring(load_config() or _get_best_keyring())
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 67, in _get_best_keyring
    keyrings = backend.get_all_keyring()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 24, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 127, in get_all_keyring
    exceptions=TypeError))
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 35, in suppress_exceptions
    for callable in callables:
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 119, in is_class_viable
    keyring_cls.priority
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/properties.py", line 22, in __get__
    return self.fget.__get__(None, owner)()
  File "/usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py", line 29, in priority
    bus = secretstorage.dbus_init()
  File "/usr/lib/python2.7/dist-packages/secretstorage/__init__.py", line 42, in dbus_init
    return dbus.SessionBus()
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 211, in __new__
    mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 100, in __new__
    bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/bus.py", line 122, in __new__
    bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusExcep...

[Terry Howe (thowe-g)]

I'm running keyring 3.4 and I do not have that line of code on line 29 of /usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py

Take a look at that file and see what line 29 says.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[zxcoolman (c-mutz)]

I have same problem with dev environnemnt.

New for this bug ?

[Terry Howe (thowe-g)]

Maybe the requirement for keyring should be bumped to 3.4

https://github.com/openstack/requirements/blob/master/global-requirements.txt#L141

[Terry Howe (thowe-g)]

https://review.openstack.org/93770

[Ritesh Paiboina (rsritesh)]

Please tell me work around for this.

[zxcoolman (c-mutz)]

sorry I forget important information . I use keyring 3.8 :
root@DevStackOSDomU:/# pip show keyring
---
Name: keyring
Version: 3.8
Location: /usr/local/lib/python2.7/dist-packages
Requires:

stack@DevStackOSDomU:/root$ keyring help
Traceback (most recent call last):
  File "/usr/local/bin/keyring", line 9, in <module>
    load_entry_point('keyring==3.8', 'console_scripts', 'keyring')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 343, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2309, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2015, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/local/lib/python2.7/dist-packages/keyring/__init__.py", line 12, in <module>
    from .core import (set_keyring, get_keyring, set_password, get_password,
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 184, in <module>
    init_backend()
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 59, in init_backend
    set_keyring(load_config() or _get_best_keyring())
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 67, in _get_best_keyring
    keyrings = backend.get_all_keyring()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 24, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 149, in get_all_keyring
    exceptions=TypeError))
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 35, in suppress_exceptions
    for callable in callables:
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 141, in is_class_viable
    keyring_cls.priority
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/properties.py", line 22, in __get__
    return self.fget.__get__(None, owner)()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/XDG.py", line 18, in wrapper
    return func(*args, **kwargs) * self.multiplier
  File "/usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py", line 31, in priority
    bus = secretstorage.dbus_init()
  File "/usr/lib/python2.7/dist-packages/secretstorage/__init__.py", line 42, in dbus_init
    return dbus.SessionBus()
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 211, in __new__
    mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 100, in __new__
    bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/bus.py", line 122, in __new__
    bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NotSupported: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11

[Terry Howe (thowe-g)]

Well, 3.4 was working for me and 3.8 is working for me now. I'm not sure how to reproduce this problem on 3.8.

[zxcoolman (c-mutz)]

I used devstack for to install openstack. I created Virtual Machine with this URL : https://github.com/openstack-dev/devstack/tree/master/tools/xen.

I Think it's same problem but with different version.

[Terry Howe (thowe-g)]

Has anyone tried setting

    export DISPLAY=:0

[zxcoolman (c-mutz)]

With keyring version 3.8
root@DevStackOSDomU:~# pip show keyring
---
Name: keyring
Version: 3.8
Location: /usr/local/lib/python2.7/dist-packages
Requires:

root@DevStackOSDomU:~# export DISPLAY=:0

root@DevStackOSDomU:~# keyring help
Traceback (most recent call last):
  File "/usr/local/bin/keyring", line 9, in <module>
    load_entry_point('keyring==3.8', 'console_scripts', 'keyring')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 343, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2309, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2015, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/local/lib/python2.7/dist-packages/keyring/__init__.py", line 12, in <module>
    from .core import (set_keyring, get_keyring, set_password, get_password,
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 184, in <module>
    init_backend()
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 59, in init_backend
    set_keyring(load_config() or _get_best_keyring())
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 67, in _get_best_keyring
    keyrings = backend.get_all_keyring()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 24, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 149, in get_all_keyring
    exceptions=TypeError))
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 35, in suppress_exceptions
    for callable in callables:
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 141, in is_class_viable
    keyring_cls.priority
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/properties.py", line 22, in __get__
    return self.fget.__get__(None, owner)()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/XDG.py", line 18, in wrapper
    return func(*args, **kwargs) * self.multiplier
  File "/usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py", line 31, in priority
    bus = secretstorage.dbus_init()
  File "/usr/lib/python2.7/dist-packages/secretstorage/__init__.py", line 42, in dbus_init
    return dbus.SessionBus()
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 211, in __new__
    mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 100, in __new__
    bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/bus.py", line 122, in __new__
    bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.Spawn.ExecFailed: //bin/dbus-launch terminated abnormally without any error message

doesn't work

[Terry Howe (thowe-g)]

I'm running Ubuntu 14.04 and I don't get that with keyring 3.8 tghrough 3.4. Either way, this is a keyring issue.

(.venv)terry@brat:~/hp/python-openstackclient$ pip show keyring
---
Name: keyring
Version: 3.8
Location: /home/terry/hp/python-openstackclient/.venv/lib/python2.7/site-packages
Requires:
(.venv)terry@brat:~/hp/python-openstackclient$ keyring help
Usage: keyring [get|set|del] SERVICE USERNAME

keyring: error: Wrong number of arguments
(.venv)terry@brat:~/hp/python-openstackclient$

[Simhon Doctori (simhond)]

Hit this problem on Devstack install using ./install_os_domU.sh for Xen, the only workaround that perfectly worked for me is following the description of : https://bugs.launchpad.net/ubuntu/+source/python-keystoneclient/+bug/1242992/comments/6

Simhon.

[taget (taget)]

I see this issue too,
my ubuntu is
Ubuntu 13.10 \n \l

taget@taget-ThinkCentre-M58p:~/code/devstack$ pip show keyring
---
Name: keyring
Version: 3.8
Location: /usr/local/lib/python2.7/dist-packages
Requires:
taget@taget-ThinkCentre-M58p:~/code/devstack$ keyring help
Traceback (most recent call last):
  File "/usr/local/bin/keyring", line 9, in <module>
    load_entry_point('keyring==3.8', 'console_scripts', 'keyring')()
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 2439, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 2155, in load
    ['__name__'])
  File "/usr/local/lib/python2.7/dist-packages/keyring/__init__.py", line 12, in <module>
    from .core import (set_keyring, get_keyring, set_password, get_password,
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 184, in <module>
    init_backend()
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 59, in init_backend
    set_keyring(load_config() or _get_best_keyring())
  File "/usr/local/lib/python2.7/dist-packages/keyring/core.py", line 67, in _get_best_keyring
    keyrings = backend.get_all_keyring()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 24, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 149, in get_all_keyring
    exceptions=TypeError))
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/__init__.py", line 35, in suppress_exceptions
    for callable in callables:
  File "/usr/local/lib/python2.7/dist-packages/keyring/backend.py", line 141, in is_class_viable
    keyring_cls.priority
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/properties.py", line 22, in __get__
    return self.fget.__get__(None, owner)()
  File "/usr/local/lib/python2.7/dist-packages/keyring/util/XDG.py", line 18, in wrapper
    return func(*args, **kwargs) * self.multiplier
  File "/usr/local/lib/python2.7/dist-packages/keyring/backends/SecretService.py", line 31, in priority
    bus = secretstorage.dbus_init()
  File "/usr/lib/python2.7/dist-packages/secretstorage/__init__.py", line 42, in dbus_init
    return dbus.SessionBus()
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 211, in __new__
    mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/_dbus.py", line 100, in __new__
    bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
  File "/usr/lib/python2.7/dist-packages/dbus/bus.py", line 122, in __new__
    bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.Spawn.ExecFailed: //bin/dbus-launch terminated abnormally with the following error: Autolaunch error: X11 initialization failed.

[taget (taget)]

Another interesting finding.

2 test cases:

1 ssh to the host to run keyring.
failed with dbus.exceptions.DBusException: org.freedesktop.DBus.Error.Spawn.ExecFailed: //bin/dbus-launch terminated abnormally with the following error: Autolaunch error: X11 initialization failed.

2 but if I access it with vnc.
I can ran keyring without errors in the console of horizon.
and I can also install devstack with that error.
but when I access dashboard.

I still can see this error e, mainloop=mainloop)
[Wed Jul 02 10:26:46.580034 2014] [:error] [pid 5248:tid 139716796188416] [remote 9.125.31.39:7010] DBusException: org.freedesktop.DBus.Error.NotSupported: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11

 2$ horizon* 3-$ g-reg 4$ g-api 5$ n-api 6$ q-svc 7$ q-agt 8$ q-dhcp 9$ q-l3 10$ q-meta 11$ n-cpu 12$ n-cond 13$ n-crt 14$ n-net

[Terry Howe (thowe-g)]

Perhaps another backend should be used other than dbus. I'm honestly not sure how to do that. Either way keyring 3.3 I think should be banned because it was insisting that DISPLAY was set.

[Clark Boylan (cboylan)]

This is still a bug... python keystoneclient does not explicitly depend on python keyring. So when you pip install python-keystoneclient it will not pull in the version of keyring that it needs. However if you have keyring installed (say because your distro gives it to you), python-keystoneclient will use that version without checking if it is the correct version.

If python-keystone client is going to depend on python keyring it should list that package as a dependency. If it is going to optionally use keyring it should double check the version of keyring that is available.

Also, maybe python-keystoneclient should just cache tokens in memory and avoid this keyring business completely.

[Terry Howe (thowe-g)]

Fair enough Clark, but I python-openstackclient is only involved because it uses python-keystoneclient, so I marked that back to invalid. Either way, I don't know the best resolution for this, so I'm bailing. Someone else is probably better suited to handle this problem.