please drop Py 2 OpenStack dependencies

[Corey Bryant]

In eoan, the python-swiftclient, python-glanceclient and python-keystoneclient binary packages have been removed. Once simplestreams drops it's Build-Depends on these packages, it should allow the python-swiftclient, python-glanceclient and pytyhon-keystoneclient source packages to migrate out of eoan-proposed.

[Dan Watkins (daniel-thewatkins)]

Diff attached which addresses the issue; also pushed up to the ubuntu/devel branch in lp:simplestreams.

[Corey Bryant (corey.bryant)]

I'm adding to this bug since there are more py2 openstack dependencies that need to be dropped.

[Corey Bryant (corey.bryant)]

I tried a build with the above named Build-Depends dropped but that results in a failure [1]. It looks like at least keystoneclient is also a run-time dependency. Can the python-simplestreams binary package be dropped at this point? That would solve that issue. I was able to successfully build the package after removing the python-simplestreams binary package and Python 2 support. Please see the attached debdiff. Note the patch should get forwarded upstream. If the approach sounds reasonable I can do that.

[1]
======================================================================
ERROR: Failure: ImportError (No module named keystoneclient.v2_0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/build/simplestreams-0.1.0-25-gba75825b/tests/unittests/test_openstack.py", line 4, in <module>
    import simplestreams.openstack as s_openstack
  File "/build/simplestreams-0.1.0-25-gba75825b/simplestreams/openstack.py", line 21, in <module>
    from keystoneclient.v2_0 import client as ksclient_v2
ImportError: No module named keystoneclient.v2_0

[Corey Bryant (corey.bryant)]

Patch for dropping py2 support.

[Scott Moser (smoser)]

Corey, simplestreams-dev

My thoughts here have a primary goal to educate the audience on the use cases
and users as I know them for simplestreams. That would then allow the teams
involved make an educated decision on how to handle python 2 support.

I do not personally have any need for python2.

A few things to note
a.) simplestreams as a project currently has packaging in its master
branch, and maintains a ubuntu/devel branch for easy uploads to ubuntu
using the same 'new-upstream-snapshot' infrastructure that cloud-init and
curtin use. Upstream debian/ directory is sometimes useful, but I think
I'd prefer to remove it entirely if it is going to differ from
ubuntu/devel.

If kept, I'd shoot for the general goal being little or no delta between
ubuntu/devel:debian/ and master:debian/ . Up until Dan's recent changes
upstream/master branch had no delta other than debian/changelog between
the ubuntu/devel and master branches in the debian/ directory.

b.) There are daily builds of simplestreams via recipe at
https://code.launchpad.net/%7Esimplestreams-dev/simplestreams/+git/simplestreams/+recipes
that include python2 support. We probably do not want to drop python2
output from those builds/ppas. I do not know of all the users of recipe
based builds/PPAs but I suspect that there are some users of python2
packaging from there (possibly from the openstack team).

c.) it looks like the trusty PPA/recipe has been failing for some time.
Since no one screamed loudly enough to fix that, and clearly no one is
listening to the recipe build failures, I'd suggest that it get dropped
https://code.launchpad.net/~simplestreams-dev/+recipe/simplestreams-daily-trusty

My suggestion would probably be to:
1. drop python2 support from upstream proper

2. Your choice on dropping the debian/ directory from upstream entirely and
maintaining it in ubuntu/devel branch or keeping the process that is there
right now.

3. add and maintain packaging branch (ubuntu/python2 ?) that kept python2
support functional for at least xenial and bionic and point the recipe
builds there.

4. add eoan recipe builds. I do not know of a way to magically ubuntu 'devel'
in recipe builds... for cloud-init and curtin, at the beginning of each ubuntu
release, someone has to go click through launchpad and add the release builds.
doc on that at https://github.com/CanonicalLtd/uss-tableflip/blob/master/doc/new-ubuntu-series.md

5. have someone pay attention to recipe-build-failure messages. This is
important as inevitably if upstream does not run python2 unit tests, then they
will start to fail at some point and this is what will tell us.

6. Fix doc at
https://github.com/CanonicalLtd/uss-tableflip/blob/master/doc/simplestreams-new-upstream-release.md
as I think it is currently wrong... Sorry for not updating it. It looks
like I moved that over to new-upstream-snapshot right before I left
Canonical.

All suggested changes should be done with merge proposals rather than diffs
attached to a bug.

[Corey Bryant (corey.bryant)]

Thanks for all the guidance Scott. I'll sync with the server team on this to figure out the path forward and will then get patches submitted in the right places.

[Christian Ehrhardt  (paelzer)]

FYI an upload of python-swiftclient [1] droppyin py2 made simplestreams a FTBFS lacking the py2 dependency.
Therefore the drop of py2 became slightly more severe now.
I'm not sure what the state of this is between Corey and Chad that seemed to be working on this.

[1]: https://launchpad.net/ubuntu/+source/python-swiftclient/1:3.8.0-0ubuntu2

[Corey Bryant (corey.bryant)]

@Christian, I haven't synced with Chad but was assumed he was focusing on it now since the bug was assigned to him. Happy to help with the patches if needed though.

[Chad Smith (chad.smith)]

Thanks for the ping on this Corey. Our plan is for me to work on this, but I've mentioned to management that we haven't had cycles to address this work over the last couple weeks. The plan is to draw up a suggestion here and work it next week at a team sprint.