strcred.py is missing from packaging on Twisted 16/Xenial

[Mike Pontillo]

While developing MAAS, I noticed the following traceback:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 579, in parseOptions
    usage.Options.parseOptions(self, options)
  File "/usr/lib/python3/dist-packages/twisted/python/usage.py", line 262, in parseOptions
    for (cmd, short, parser, doc) in self.subCommands:
  File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 596, in subCommands
    for plug in sorted(plugins, key=attrgetter('tapname')):
  File "/usr/lib/python3/dist-packages/twisted/plugin.py", line 213, in getPlugins
    allDropins = getCache(package)
--- <exception caught here> ---
  File "/usr/lib/python3/dist-packages/twisted/plugin.py", line 171, in getCache
    provider = pluginModule.load()
  File "/usr/lib/python3/dist-packages/twisted/python/modules.py", line 389, in load
    return self.pathEntry.pythonPath.moduleLoader(self.name)
  File "/usr/lib/python3/dist-packages/twisted/python/reflect.py", line 303, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/usr/lib/python3/dist-packages/twisted/python/reflect.py", line 250, in _importAndCheckStack
    reraise(excValue, excTraceback)
  File "/usr/lib/python3/dist-packages/twisted/python/compat.py", line 412, in reraise
    raise exception.with_traceback(traceback)
  File "/usr/lib/python3/dist-packages/twisted/plugins/cred_sshkeys.py", line 15, in <module>
    from twisted.cred.strcred import ICheckerFactory
builtins.ImportError: No module named 'twisted.cred.strcred'

It looks like this file exists in the upstream distribution, but not in the .deb:

$ dpkg -L python3-twisted | grep strcred
/usr/lib/python3/dist-packages/twisted/cred/test/test_strcred.py

$ apt-get source python3-twisted
$ cd twisted-16.0.0~pre1
$ find . | grep strcred
./twisted/cred/strcred.py
./twisted/cred/test/test_strcred.py

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: python3-twisted 16.0.0~pre1-1
ProcVersionSignature: Ubuntu 4.4.0-4.19-generic 4.4.1
Uname: Linux 4.4.0-4-generic x86_64
ApportVersion: 2.20-0ubuntu3
Architecture: amd64
CurrentDesktop: Unity
Date: Tue Feb 23 11:51:23 2016
PackageArchitecture: all
SourcePackage: twisted
UpgradeStatus: No upgrade log present (probably fresh install)

[Matthias Klose (doko)]

this usually means that the corresponding twisted module is not yet ported to python3. or was this already provided by an earlier version?

[Mike Pontillo (mpontillo)]

I'm not certain about Twisted, but what I can tell you is:

(1) Looking at Twisted 15.5.0, this module exists (and has roughly as much usage according to my grep)
(2) Well, we didn't see that traceback with the previous version of Twisted that was in Xenial (15.5)
(2a) ... but then again, MAAS doesn't directly use that module either; it seems the error surfaces when Twisted includes the cred_sshkeys plugin.

So either:
 - strcred module also didn't exist in the previous version of the python3 package, BUT Twisted never tried to load a plugin that depends on it
    OR
 - the strcred module existed in the previous version of python3-twisted, which is why we didn't notice this.

Either way, I think it's a regression.

I can reproduce it outside of MAAS by typing "twistd3" on the command line and then scrolling up.

[Mike Pontillo (mpontillo)]

I just checked the 15.5 package and the cred.strcred doesn't exist there, either.

It looks like this started occurring because this file is now included in packaging, where it wasn't before:

/usr/lib/python3/dist-packages/twisted/plugins/cred_sshkeys.py

So either it actually shouldn't be included, or twisted.cred.strcred has been fixed for python3 and just not included in the package. (for what it's worth, I ran it through a python3 compile and it worked, but I'm not sure what the requirements are - perhaps it didn't pass a test? Perhaps cred_sshkeys is included because it passes the test, but only with the [broken] strcred module.)

[Launchpad Janitor (janitor)]

This bug was fixed in the package twisted - 16.0.0~pre1-1ubuntu1

---------------
twisted (16.0.0~pre1-1ubuntu1) xenial; urgency=medium

  * Non-maintainer upload.
  * Fix issue #8216, add twisted.cred.strcred for Python3 installations.
    LP: #1548972.

 -- Matthias Klose <email address hidden> Wed, 24 Feb 2016 12:15:53 +0100