[SRU] Please backport python3-macaroonbakery 0.0.6-1 [universe] from bionic

Bug #1735160 reported by Andrea Azzarone on 2017-11-29
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
httmock (Ubuntu)
Undecided
Andrea Azzarone
Xenial
Undecided
Andrea Azzarone
Artful
Undecided
Unassigned
protobuf (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Andrea Azzarone
Artful
Undecided
Unassigned
py-macaroon-bakery (Ubuntu)
Undecided
Andrea Azzarone
Xenial
Undecided
Andrea Azzarone
Artful
Undecided
Andrea Azzarone
pymacaroons (Ubuntu)
Undecided
Unassigned
Xenial
Undecided
Andrea Azzarone
Artful
Undecided
Andrea Azzarone

Bug Description

[Impact]
As part of allowing Ubuntu users to enable canonical-livepatch from software-properties GUI (https://wiki.ubuntu.com/SoftwareUpdates#Update_settings) we need to backport python3-macaroonbakery 0.0.6-1 [universe] from bionic. This will requires quite few changes:

- backport httmock 1.2.6-1 [universe] from bionic - no httmock in xenial
- backport pymacaroons 0.12.0-1 [universe] from bionic - xenial has 0.9.2-0ubuntu1
- SRU some changes in google-apputils-python - https://bugs.launchpad.net/ubuntu/+source/google-apputils-python/+bug/1735162
- add python3-protobuf to python-protobuf 2.6.1-1.3 - Right now the python3 package is not built.

[Test case]
- for protobuf: $ python3 -c "import google.protobuf"
- for protobuf: $ python2 -c "import google.protobuf"
- for python3-macaroonbakery: make sure all the tests pass
- make sure "snapcraft login" works properly

[Regression Potential]
- httmock, none has it's not in xenial
- protobuf, reduced considering that the code has been backported from master (small changes to make it compatabile py3 compatabile)
- pymacaroons, none has 0.12 is backward compatible with 0.9.2
- py-macaroon-bakery, none has it's not in xenial.

Andrea Azzarone (azzar1) on 2017-11-29
description: updated
Andrea Azzarone (azzar1) wrote :

Please find attached the proposed debdiff to add python3-protobuf.

Andrea Azzarone (azzar1) wrote :
Andrea Azzarone (azzar1) on 2017-12-11
no longer affects: xenial-backports
summary: - Please backport python3-macaroonbakery 0.0.6-1 [universe] from bionic
+ [SRU] Please backport python3-macaroonbakery 0.0.6-1 [universe] from
+ bionic
description: updated
description: updated
description: updated
description: updated

The attachment "protobuf.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Andrea Azzarone (azzar1) on 2017-12-13
description: updated
Andrea Azzarone (azzar1) on 2017-12-13
description: updated
Andrea Azzarone (azzar1) wrote :

py-macaroon-bakery cannot be imported as it is (otherwise we'll need to update the version of protobuf and python3-nacl in xenial, breaking all applications already using it). Please find attached the proposed debdiff.

description: updated
description: updated
Simon Quigley (tsimonq2) wrote :

I feel like an ack/nack from the SRU Team before sponsoring would be preferred, given that this is a bit of a special case. Subscribed.

Iain Lane (laney) wrote :

Andy asked me to have a look, so here's my initial feedback.

General
=======

I'm not sure if these changes should be backported to Artful too, to maintain the upgrade path for users transitioning off Xenial. Could you please ask the SRU team?

Otherwise, the changes are risky but I think they should be in principle okay.

The test cases and regression potential in the description here could do with fleshing out a bit - could you do that please? For example, snapcraft depends on pymacaroons - probably want to make sure that keeps working. And it is supposed to be compatible, but that doesn't mean there is no bug in the newer version - it is a regression potential IMO.

httmock
=======

Since this is completely new and a no-change backport, I think it's okay. Uploaded.

protobuf
========

I'm a bit worried about bugs coming from the use of 2to3 which is best used as a starting point for a manual port, not for an automatic conversion - how about taking the upstream patches instead?

  https://github.com/google/protobuf/commit/f336d4b7a5c1d369ed508e513d482c885705e939

  https://github.com/google/protobuf/commit/fe7d9379df3ce7c951bc0652a451413cff02382a

  https://github.com/google/protobuf/commit/47ee4d37c17db8e97fe5b15cf918ab56ff93bb18

I tried the testsuite with this and it works after resolving some minor conflicts.

pymacaroons
===========

Seems okay if it's compatible - can you check with Colin please? I'm happy to handle the actual upload.

py-macaroon-bakery
==================

I'm not qualified to review this - maybe also check with Colin as the maintainer? The patch says it's editing a generated file - is that right?

Changed in httmock (Ubuntu):
status: New → In Progress
assignee: nobody → Andrea Azzarone (azzar1)
status: In Progress → Fix Released
Changed in httmock (Ubuntu Xenial):
status: New → In Progress
assignee: nobody → Andrea Azzarone (azzar1)
Colin Watson (cjwatson) wrote :

pymacaroons does have one potentially-interesting backward-incompatible change, which is that Caveat.caveat_id and Macaroon.identifier are now bytes rather than str. I think the only place this is at all likely to cause a problem in xenial is in snapcraft, and it's definitely worth checking whether "snapcraft login" still works.

The py-macaroon-bakery change looks OK-ish, as long as (obviously) somebody confirms that it's wire-compatible with the servers this is intended to talk to.

Download full text (4.1 KiB)

"snapcraft login" seems to work fine here.

On Tue, Jan 9, 2018 at 4:02 PM, Colin Watson <email address hidden> wrote:

> pymacaroons does have one potentially-interesting backward-incompatible
> change, which is that Caveat.caveat_id and Macaroon.identifier are now
> bytes rather than str. I think the only place this is at all likely to
> cause a problem in xenial is in snapcraft, and it's definitely worth
> checking whether "snapcraft login" still works.
>
> The py-macaroon-bakery change looks OK-ish, as long as (obviously)
> somebody confirms that it's wire-compatible with the servers this is
> intended to talk to.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1735160
>
> Title:
> [SRU] Please backport python3-macaroonbakery 0.0.6-1 [universe] from
> bionic
>
> Status in httmock package in Ubuntu:
> Fix Released
> Status in protobuf package in Ubuntu:
> New
> Status in pymacaroons package in Ubuntu:
> New
> Status in python-nacl package in Ubuntu:
> New
> Status in httmock source package in Xenial:
> In Progress
> Status in protobuf source package in Xenial:
> New
> Status in pymacaroons source package in Xenial:
> New
> Status in python-nacl source package in Xenial:
> New
>
> Bug description:
> [Impact]
> As part of allowing Ubuntu users to enable canonical-livepatch from
> software-properties GUI (https://wiki.ubuntu.com/SoftwareUpdates#Update_
> settings) we need to backport python3-macaroonbakery 0.0.6-1 [universe]
> from bionic. This will requires quite few changes:
>
> - backport httmock 1.2.6-1 [universe] from bionic - no httmock in xenial
> - backport pymacaroons 0.12.0-1 [universe] from bionic - xenial has
> 0.9.2-0ubuntu1
> - SRU some changes in google-apputils-python -
> https://bugs.launchpad.net/ubuntu/+source/google-
> apputils-python/+bug/1735162
> - add python3-protobuf to python-protobuf 2.6.1-1.3 - Right now the
> python3 package is not built.
>
> [Test case]
> - for python3-protobuf: $ python3 -c "import google.protobuf"
> - for python3-macaroonbakery: make sure all the tests pass
>
> [Regression Potential]
> - httmock, none has it's not in xenial
> - python3-protobuf, none has the python2 code was not edited
> TBC
> - pymacaroons, none has 0.12 is backward compatible with 0.9.2
> - python3-protobuf, none has it's not in xenial.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/httmock/+bug/
> 1735160/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: distribution=ubuntu; sourcepackage=httmock;
> component=universe; status=Fix Released; importance=Undecided; assignee=
> <email address hidden>;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=protobuf;
> component=main; status=New; importance=Undecided; assignee=None;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=pymacaroons;
> component=universe; status=New; importance=Undecided; assignee=None;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=python-nacl;
> component=universe; status=New; importance=Undecided; assignee=None;
> Launchpad-Bug: distribution=ubunt...

Read more...

Andrea Azzarone (azzar1) wrote :
Download full text (6.3 KiB)

On Tue, Jan 9, 2018 at 12:38 PM, Iain Lane <email address hidden> wrote:

> Andy asked me to have a look, so here's my initial feedback.
>
> General
> =======
>
> I'm not sure if these changes should be backported to Artful too, to
> maintain the upgrade path for users transitioning off Xenial. Could you
> please ask the SRU team?
>

At least for our usecase there is no need to backport this to Artful. Who
should I ask?

>
> Otherwise, the changes are risky but I think they should be in principle
> okay.
>
> The test cases and regression potential in the description here could do
> with fleshing out a bit - could you do that please? For example,
> snapcraft depends on pymacaroons - probably want to make sure that keeps
> working. And it is supposed to be compatible, but that doesn't mean
> there is no bug in the newer version - it is a regression potential IMO.
>

I tested snapcraft-login as in above question.

>
> httmock
> =======
>
> Since this is completely new and a no-change backport, I think it's
> okay. Uploaded.
>
> protobuf
> ========
>
> I'm a bit worried about bugs coming from the use of 2to3 which is best
> used as a starting point for a manual port, not for an automatic
> conversion - how about taking the upstream patches instead?
>
> https://github.com/google/protobuf/commit/f336d4b7a5c1d369ed508e513d482c
> 885705e939
>
> https://github.com/google/protobuf/commit/fe7d9379df3ce7c951bc0652a45141
> 3cff02382a
>
> https://github.com/google/protobuf/commit/47ee4d37c17db8e97fe5b15cf918ab
> 56ff93bb18
>
> I tried the testsuite with this and it works after resolving some minor
> conflicts.
>

I'll take a look into it.

>
> pymacaroons
> ===========
>
> Seems okay if it's compatible - can you check with Colin please? I'm
> happy to handle the actual upload.
>
> py-macaroon-bakery
> ==================
>
> I'm not qualified to review this - maybe also check with Colin as the
> maintainer? The patch says it's editing a generated file - is that
> right?
>

It's editing the protobuf generated file. It seems like these changes are
wire-compatible with the livepatch servers".

>
> ** Changed in: httmock (Ubuntu)
> Status: New => In Progress
>
> ** Changed in: httmock (Ubuntu)
> Assignee: (unassigned) => Andrea Azzarone (azzar1)
>
> ** Also affects: protobuf (Ubuntu Xenial)
> Importance: Undecided
> Status: New
>
> ** Also affects: python-nacl (Ubuntu Xenial)
> Importance: Undecided
> Status: New
>
> ** Also affects: pymacaroons (Ubuntu Xenial)
> Importance: Undecided
> Status: New
>
> ** Also affects: httmock (Ubuntu Xenial)
> Importance: Undecided
> Status: New
>
> ** Changed in: httmock (Ubuntu)
> Status: In Progress => Fix Released
>
> ** Changed in: httmock (Ubuntu Xenial)
> Status: New => In Progress
>
> ** Changed in: httmock (Ubuntu Xenial)
> Assignee: (unassigned) => Andrea Azzarone (azzar1)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1735160
>
> Title:
> [SRU] Please backport python3-macaroonbakery 0.0.6-1 [universe] from
> bionic
>
> Status in httmock package in Ubuntu:
>...

Read more...

Andrea Azzarone (azzar1) wrote :

Splitted in 3 patches.

Iain Lane (laney) wrote :

Uploaded protobuf to the Xenial SRU queue. I fixed some things from your diff:

  - Added (LP: #xxx) to the changelog so that it's linked to this bug
  - Added --with python3 in debian/rules. Without this we don't get the right python3 dependencies on the pkg.
  - Dropped the add-python3.patch file, as we `cp python python3' in rules.
  - Added python3-six build dependency (something else pulled this in already so this isn't that critical)

> At least for our usecase there is no need to backport this to Artful. Who
> should I ask?

Someone in the SRU team (~ubuntu-sru). The problem is that if you upgrade from Xenial to Zesty or (soon) Artful and we haven't SRUed the fixes then users could either become unfixed or in some cases (didn't check / don't remember for this bug) package versions might go backwards. I'm just not sure what the SRU policy on this is. (For backports the policy is that you have to backport to all of the intermediate releases too, for these reasons.)

> I tested snapcraft-login as in above question.

Great - I just think that should be in the SRU description too so that the actual uploads make sure Snapcraft still works.

affects: python-nacl (Ubuntu) → py-macaroon-bakery (Ubuntu)
Changed in protobuf (Ubuntu):
status: New → Fix Released
Changed in protobuf (Ubuntu Xenial):
status: New → In Progress
Changed in py-macaroon-bakery (Ubuntu):
assignee: nobody → Andrea Azzarone (azzar1)
Changed in protobuf (Ubuntu Xenial):
assignee: nobody → Andrea Azzarone (azzar1)
Changed in py-macaroon-bakery (Ubuntu):
status: New → Fix Released
Changed in pymacaroons (Ubuntu):
status: New → Fix Released
Andrea Azzarone (azzar1) on 2018-01-11
description: updated
description: updated
Andrea Azzarone (azzar1) wrote :

Oh I removed the python3 patch from the series but the not actual file.

Regarding 17.10 sil2100 confirmed me that this is required. But at least it's going to be easier considering that:

- python3-httmock is already in artful (1.2.6-1 universe)
- python3-protobuf is already in artful (3.0.0-9ubuntu5 universe) (so no need to apply the nacl patch in py-macaroon-bakery)
- python3-nacl's version of artful is 1.1.1-1build2 (so no need to apply the nacl patch in py-macaroon-bakery)

The only things to be done are:
- target https://bugs.launchpad.net/ubuntu/+source/google-apputils-python/+bug/1735162 for 17.10 too
- relax python3-six deps in py-macaroon-bakery
- SRU python3-pymacaroons

Iain Lane (laney) wrote :

Uploaded py-macaroon-bakery and pymacaroons to the queue (unapproved and new respectively). I'll look at the 17.10 stuff after lunch.

Changed in pymacaroons (Ubuntu Xenial):
status: New → In Progress
Changed in py-macaroon-bakery (Ubuntu Xenial):
status: New → In Progress
assignee: nobody → Andrea Azzarone (azzar1)
Changed in pymacaroons (Ubuntu Xenial):
assignee: nobody → Andrea Azzarone (azzar1)
Iain Lane (laney) wrote :

Should all be uploaded now, thanks! Please check what I did for py-macaroon-bakery in artful (https://launchpadlibrarian.net/353259789/py-macaroon-bakery_0.0.6-1_0.0.6-1~ubuntu17.10.1.diff.gz) in case I messed it up somehow.

Cheers!

Changed in httmock (Ubuntu Artful):
status: New → Invalid
Changed in protobuf (Ubuntu Artful):
status: New → Invalid
Changed in pymacaroons (Ubuntu Artful):
assignee: nobody → Andrea Azzarone (azzar1)
status: New → In Progress
Changed in py-macaroon-bakery (Ubuntu Artful):
assignee: nobody → Andrea Azzarone (azzar1)
status: New → In Progress
Robie Basak (racb) wrote :

> The problem is that if you upgrade from Xenial to Zesty or (soon) Artful and we haven't SRUed the fixes then users could either become unfixed or in some cases (didn't check / don't remember for this bug) package versions might go backwards.

Worse, a user may not receive future security updates for the package in Artful.

Iain Lane (laney) wrote :

On Wed, Jan 17, 2018 at 05:05:07PM -0000, Robie Basak wrote:
> > The problem is that if you upgrade from Xenial to Zesty or (soon)
> Artful and we haven't SRUed the fixes then users could either become
> unfixed or in some cases (didn't check / don't remember for this bug)
> package versions might go backwards.
>
> Worse, a user may not receive future security updates for the package in
> Artful.

No need to argue, since we SRUed to Artful.

--
Iain Lane [ <email address hidden> ]
Debian Developer [ <email address hidden> ]
Ubuntu Developer [ <email address hidden> ]

Andrea Azzarone (azzar1) wrote :

Any news on this?

Hello Andrea, or anyone else affected,

Accepted pymacaroons into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pymacaroons/0.12.0-1~ubuntu17.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in pymacaroons (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Andrea Azzarone (azzar1) wrote :

python3-pymacaroons works properly, but python3-macaroonbakery is not in proposed yet so I cannot mark this bug as verification-done.

tags: added: verification-failed-artful
removed: verification-needed-artful
Łukasz Zemczak (sil2100) wrote :

@azzar1 You marked this bug as verification-failed-artful - does it mean the pymacaroons package is broken? Usually when a package is marked as -failed it is a sign for the SRU/AA team to remove the package from -proposed - is this the case here? Your comment implies as if we're just waiting for another part of the fix instead.

Andrea Azzarone (azzar1) wrote :

@sil2100 pymacaroons packages works fine. But can you please take care of the other ones too. They've been waiting in the queue for quite some time.

tags: added: verification-needed-artful
removed: verification-failed-artful
Łukasz Zemczak (sil2100) wrote :

Ok, this bug is confusing me. I do not see py-macaroon-bakery in either the xenial nor artful queues. There is also no httmock for xenial as well. I can take a look at the xenial pymacaroons and protobuf, but I guess we're still missing the py-macaroon-bakery uploads which seem to be very important.

What's the status of those? Did they get uploaded and rejected? Can someone re-upload?

Łukasz Zemczak (sil2100) wrote :

Hello Andrea, or anyone else affected,

Accepted protobuf into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/protobuf/2.6.1-1.3ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in protobuf (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Łukasz Zemczak (sil2100) wrote :

Ok, those are NEW packages. Just by looking at the UNAPPROVED queue and the bug description it was not clear (although I should have just looked up the source - but on the other hand there was no reason if I didn't see an upload). Maybe it would be best if it was made clear that py-macaroon-bakery is a NEW package for those series. Anyway, this is now noted. Thanks!

Łukasz Zemczak (sil2100) wrote :

Hello Andrea, or anyone else affected,

Accepted pymacaroons into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pymacaroons/0.12.0-1~ubuntu16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in pymacaroons (Ubuntu Xenial):
status: In Progress → Fix Committed
Łukasz Zemczak (sil2100) wrote :

@azzar1 the protobuf package I have accepted into xenial-proposed FTFBS for some architectures. Could you take a look at that? I didn't look at those yet at all, maybe they just need a re-build.

https://launchpad.net/ubuntu/+source/protobuf/2.6.1-1.3ubuntu1

Once it's built for all arches I can approve the new python3 binaries.

Andrea Azzarone (azzar1) wrote :

@sil2100 please find attached an updated debdiff for protobuf. I just replaced this line:
TestGetAndDeserialize('optional_uint32', 1 << 31, long)

with:
TestGetAndDeserialize('optional_uint32', 1 << 31, integer_64)

as upstream already did: https://github.com/google/protobuf/blob/master/python/google/protobuf/internal/reflection_test.py

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers