Pip crashes with cryptography/cffi/openssl error

[lusepuster]

Traceback when trying to run any pip command:

Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 558, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2682, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2355, in load
    return self.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2361, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 74, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar # noqa
  File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/lib/python2.7/dist-packages/pip/download.py", line 22, in <module>
    import requests, six
  File "/usr/lib/python2.7/dist-packages/requests/__init__.py", line 53, in <module>
    from .packages.urllib3.contrib import pyopenssl
  File "/usr/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 53, in <module>
    import OpenSSL.SSL
  File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import rand, crypto, SSL
  File "/usr/lib/python2.7/dist-packages/OpenSSL/rand.py", line 11, in <module>
    from OpenSSL._util import (
  File "/usr/lib/python2.7/dist-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/usr/lib/python2.7/dist-packages/cryptography/hazmat/bindings/openssl/binding.py", line 11, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
AttributeError: 'module' object has no attribute '_init_cffi_1_0_external_module'

ProblemType: Bug
DistroRelease: Ubuntu 15.10
Package: python-pip 1.5.6-7ubuntu1
ProcVersionSignature: Ubuntu 4.2.0-16.19-generic 4.2.3
Uname: Linux 4.2.0-16-generic x86_64
NonfreeKernelModules: wl
ApportVersion: 2.19.1-0ubuntu4
Architecture: amd64
CurrentDesktop: GNOME
Date: Tue Nov 3 17:43:31 2015
DistributionChannelDescriptor:
 # This is a distribution channel descriptor
 # For more information see http://wiki.ubuntu.com/DistributionChannelDescriptor
 canonical-oem-somerville-precise-amd64-20130203-1
InstallationDate: Installed on 2014-01-30 (641 days ago)
InstallationMedia: Ubuntu 12.04 "Precise" - Build amd64 LIVE Binary 20130203-13:50
PackageArchitecture: all
SourcePackage: python-pip
UpgradeStatus: Upgraded to wily on 2015-10-28 (6 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Ross Patterson (rossp)]

I was able to work around this based on the following comment by upgrading the OS package that installs the Python cffi package with `$ sudo easy_install -U cffi`:

https://bugs.launchpad.net/devstack/+bug/1484870/comments/2

[Battant (mparchet)]

Starting LSB: Calendar and Contacts Server...
 calendarserver[2071]: Unhandled Error
 calendarserver[2071]: Traceback (most recent call last):
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 579,
 calendarserver[2071]: usage.Options.parseOptions(self, options)
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 262, in
 calendarserver[2071]: for (cmd, short, parser, doc) in self.subCommands:
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 596,
 calendarserver[2071]: for plug in sorted(plugins, key=attrgetter('tapname')):
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/plugin.py", line 213, in getPl
 calendarserver[2071]: allDropins = getCache(package)
 calendarserver[2071]: --- <exception caught here> ---
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/plugin.py", line 171, in getCa
 calendarserver[2071]: provider = pluginModule.load()
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/python/modules.py", line 389,
 calendarserver[2071]: return self.pathEntry.pythonPath.moduleLoader(self.name)
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 303,
 calendarserver[2071]: topLevelPackage = _importAndCheckStack(trialname)
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 242,
 calendarserver[2071]: return __import__(importName)
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/plugins/twisted_core.py", line
 calendarserver[2071]: from twisted.internet.endpoints import (
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/internet/endpoints.py", line 4
 calendarserver[2071]: from twisted.protocols.tls import TLSMemoryBIOFactory
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/twisted/protocols/tls.py", line 41, in
 calendarserver[2071]: from OpenSSL.SSL import Error, ZeroReturnError, WantReadError
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <modu
 calendarserver[2071]: from OpenSSL import rand, crypto, SSL
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/OpenSSL/rand.py", line 11, in <module>
 calendarserver[2071]: from OpenSSL._util import (
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/OpenSSL/_util.py", line 6, in <module>
 calendarserver[2071]: from cryptography.hazmat.bindings.openssl.binding import Binding
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/cryptography/hazmat/bindings/openssl/b
 calendarserver[2071]: error=-1)
 calendarserver[2071]: File "/usr/lib/python2.7/dist-packages/cryptography/hazmat/bindings/openssl/b
 calendarserver[2071]: ffi.def_extern(name=name, **kwargs)(func)
 calendarserver[2071]: exceptions.AttributeError: 'CompiledFFI' object has no attribute 'def_extern'

My configuration :

Ubuntu 16.04 LTS 64 bit

 uname -a
Linux mparchet--4.4.0-22-generic #39-Ubuntu SMP Thu May 5 16:53:32 UTC 2016 x...

[Battant (mparchet)]

Also affect :

https://bugs.launchpad.net/ubuntu/+source/calendarserver/+bug/1361130

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Battant (mparchet)]

Hello,

Because python-cryptography dont't work, calendarserver don't work.

arserver[2089]: Traceback (most recent call last):
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 579, in parseOptio
arserver[2089]: usage.Options.parseOptions(self, options)
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 262, in parseOptions
arserver[2089]: for (cmd, short, parser, doc) in self.subCommands:
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 596, in subCommand
arserver[2089]: for plug in sorted(plugins, key=attrgetter('tapname')):
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/plugin.py", line 213, in getPlugins
arserver[2089]: allDropins = getCache(package)
arserver[2089]: --- <exception caught here> ---
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/plugin.py", line 171, in getCache
arserver[2089]: provider = pluginModule.load()
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/python/modules.py", line 389, in load
arserver[2089]: return self.pathEntry.pythonPath.moduleLoader(self.name)
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 303, in namedAny
arserver[2089]: topLevelPackage = _importAndCheckStack(trialname)
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 242, in _importAndC
arserver[2089]: return __import__(importName)
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/plugins/twisted_core.py", line 5, in <module
arserver[2089]: from twisted.internet.endpoints import (
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/internet/endpoints.py", line 48, in <module>
arserver[2089]: from twisted.protocols.tls import TLSMemoryBIOFactory
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/protocols/tls.py", line 41, in <module>
arserver[2089]: from OpenSSL.SSL import Error, ZeroReturnError, WantReadError
arserver[2089]: File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module>
arserver[2089]: from OpenSSL import rand, crypto, SSL
arserver[2089]: File "/usr/lib/python2.7/dist-packages/OpenSSL/rand.py", line 11, in <module>
 sam. 2016-05-14 21:09:01 CEST. --
d[1]: Starting LSB: Calendar and Contacts Server...
arserver[2089]: Unhandled Error
arserver[2089]: Traceback (most recent call last):
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 579, in parseOptio
arserver[2089]: usage.Options.parseOptions(self, options)
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/python/usage.py", line 262, in parseOptions
arserver[2089]: for (cmd, short, parser, doc) in self.subCommands:
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/application/app.py", line 596, in subCommand
arserver[2089]: for plug in sorted(plugins, key=attrgetter('tapname')):
arserver[2089]: File "/usr/lib/python2.7/dist-packages/twisted/plugin.py", line 213, in getPlugins
arserver[2089]: allDropins = getCache(package)
arserver[208...

[Battant (mparchet)]

see also

https://bugs.launchpad.net/ubuntu/+source/python-cryptography/+bug/1555723

Best regards

Battant

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Barry Warsaw (barry)]

I'm still unable to reproduce this. Can someone please take me through exactly the steps you're using that trigger this bug? Please include a list of your installed packages, or take me through the steps from a live build or clean install of 16.04. If you're *not* using amd64, please let me know.

Clearly there's something going on, likely some kind of bad package interaction, but I can't reproduce it locally.

[Battant (mparchet)]

 Hello ,

 Thanks rossp, With this, command line bellow The calendar server seems not show any trace it errer but don't walk

sudo easy_install -U cffi

  Best regards

Battant

[Battant (mparchet)]

Hello,

see attachment my installed package list.

could you help me please to solve this issues ?

Best regards

Battant

[Barry Warsaw (barry)]

What if you use `pip install` instead of `easy_install`? Can you narrow down the package list? I still need a reproducible recipe.

[Battant (mparchet)]

Hello,

Very sory for my late reply I missed your message.

Finally, because calendarserver requires memcached, I have installed the calendarserver package using the ubuntu package provided by ubuntu

apt-cache policy calendarserver
calendarserver:
  Installed: 7.0+dfsg-2
  Candidate: 7.0+dfsg-2
  Version table:
 *** 7.0+dfsg-2 500
        500 http://ch.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        100 /var/lib/dpkg/status

When I followed the documentation :

/usr/share/doc/calendarserver/README.Debian

I got the following results :

1. calendarserver service seems running but only for five minutes (the it exits)
2. During those five miuntes, I cannot contact it.

Any comment and suggestion would be very much appreciated

I will check carefully my mailbox.

Best regards

Battant

[Otus (jan-varho)]

I was getting a similar error on 16.04 by running "python -m pip", but *not* when running "pip" directly. Upgrading CFFI per comment #3 fixed it.

This is with a system upgraded from Trusty (and before) in case that matters.

[Neal McBurnett (nealmcb)]

I have this issue in Bionic. It was caused by pip using an old install of an obsolete cffi in ~/.local from a pip --user install, as explained (and diagnosed) here:

    https://stackoverflow.com/q/54054263/507544

[Kai Kasurinen (kai-kasurinen)]

Since it seems likely to me that this is a local configuration problem, rather than a bug in Ubuntu, I am marking this bug as 'Incomplete'.

However, if you believe that this is really a bug in Ubuntu, then we would be grateful if you would provide a more complete description of the problem with steps to reproduce, explain why you believe this is a bug in Ubuntu rather than a problem specific to your system, and then change the bug status back to "New".

For local configuration issues, you can find assistance here:
http://www.ubuntu.com/support/community

[Launchpad Janitor (janitor)]

[Expired for python-pip (Ubuntu) because there has been no activity for 60 days.]