Ubuntu Distributed Development

users are not warned when branching ubuntu:foo (or lp:ubuntu/foo) and the package import of foo is out of date

Reported by Barry Warsaw on 2010-07-23
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Bazaar
High
John A Meinel
2.1
High
John A Meinel
2.2
High
John A Meinel
2.3
High
John A Meinel
2.4
High
John A Meinel
Launchpad itself
High
Unassigned
Ubuntu Distributed Development
High
John A Meinel
bzr (Ubuntu)
High
Unassigned
Maverick
Undecided
Unassigned

Bug Description

When the importer has been failing, it would be nice if 'bzr branch ubuntu:foo' (a.k.a. lp:ubuntu/foo) warned you that you were getting an outdated source branch. It might also print out the apt-get source command to get the updated source.

Please also refer to bug 609186 for possibly related ui requests.

James Westby (james-w) on 2010-07-23
Changed in udd:
status: New → Triaged
importance: Undecided → High
Martin Pool (mbp) wrote :

One way to fix this is: add a 'motd' field to the bzr metadata that's shown when you fetch or update a branch; set that from the package importer; perhaps also show it in bzr.

tags: added: confusing-ui udd
Changed in udd:
assignee: nobody → canonical-bazaar (canonical-bazaar)
Martin Pool (mbp) wrote :

bug 121201 proposed adding a motd capability to bzr.

I'll also mark this against bzr and launchpad as it may need changes there (or it may not.)

Changed in launchpad:
status: New → Triaged
importance: Undecided → High
Changed in bzr:
importance: Undecided → High
John A Meinel (jameinel) wrote :

Some similarity with bug #121201, though that seems to be more about a specific server-side message, which seems independent of specific branches, etc.

Martin Pool (mbp) on 2011-02-09
Changed in bzr:
status: New → Triaged
Vincent Ladeuil (vila) on 2011-02-10
Changed in bzr:
status: Triaged → Confirmed
Barry Warsaw (barry) on 2011-03-11
summary: - bzr branch lp:ubuntu/foo should warn when foo is out of date
+ bzr branch ubuntu:foo (a.k.a. lp:ubuntu/foo) should warn when foo is out
+ of date
description: updated
summary: - bzr branch ubuntu:foo (a.k.a. lp:ubuntu/foo) should warn when foo is out
- of date
+ users are not warned when branching ubuntu:foo (or lp:ubuntu/foo) and
+ the package import of foo is out of date
Jelmer Vernooij (jelmer) wrote :

Related to bug 362619 and bug 362620

We discussed this a bit yesterday. One option would be to add some
client side code that asks Launchpad what is the current published
version of a package, and then we can check whether the branch is up
to date without needing to change the package importer itself or
launchpad. This is perhaps also more likely to be accurate in the
case where the package importer is failing since it checks through
entirely independent means.

James Westby (james-w) wrote :

On Thu, 19 May 2011 13:11:53 -0000, Martin Pool <email address hidden> wrote:
> We discussed this a bit yesterday. One option would be to add some
> client side code that asks Launchpad what is the current published
> version of a package, and then we can check whether the branch is up
> to date without needing to change the package importer itself or
> launchpad. This is perhaps also more likely to be accurate in the
> case where the package importer is failing since it checks through
> entirely independent means.

I think that's a good idea.

My only concern would be the possibility that we change the definition
of "current" for e.g. natty-proposed. We don't want to be restricted in
what we can do there due to older clients.

I'm not sure what we can do there without adding an API to LP
specifically for this.

Thanks,

James

Martin Pool (mbp) wrote :

On 19 May 2011 19:22, James Westby <email address hidden> wrote:
> On Thu, 19 May 2011 13:11:53 -0000, Martin Pool <email address hidden> wrote:
>> We discussed this a bit yesterday.  One option would be to add some
>> client side code that asks Launchpad what is the current published
>> version of a package, and then we can check whether the branch is up
>> to date without needing to change the package importer itself or
>> launchpad.  This is perhaps also more likely to be accurate in the
>> case where the package importer is failing since it checks through
>> entirely independent means.
>
> I think that's a good idea.
>
> My only concern would be the possibility that we change the definition
> of "current" for e.g. natty-proposed. We don't want to be restricted in
> what we can do there due to older clients.
>
> I'm not sure what we can do there without adding an API to LP
> specifically for this.

I think we can write a generic check "is the latest tag (or changelog
entry) in $branch up to date with the latest publication of the
corresponding package in $distroseries." To start with I would
probably just bind that to a command that people can run to check.

There is then a bit of a question about how we determine which series
matches which distroseries, and when to test this. I would think
about testing it when looking at the tip of an ubuntu-namespace branch
on Launchpad. I'm pretty sure there are lp:ubuntu/natty-proposed/foo
branches, and that would be when we check -proposed.

James Westby (james-w) wrote :

On Fri, 20 May 2011 09:07:51 -0000, Martin Pool <email address hidden> wrote:
> I think we can write a generic check "is the latest tag (or changelog
> entry) in $branch up to date with the latest publication of the
> corresponding package in $distroseries." To start with I would
> probably just bind that to a command that people can run to check.
>
> There is then a bit of a question about how we determine which series
> matches which distroseries, and when to test this. I would think
> about testing it when looking at the tip of an ubuntu-namespace branch
> on Launchpad. I'm pretty sure there are lp:ubuntu/natty-proposed/foo
> branches, and that would be when we check -proposed.

Right, but -proposed behaves a little oddly itself. It may be that we
change the -proposed branches to match -release branches at release
time. Also, packages can be deleted from -proposed, and it's not clear
if we should uncommit, or perform an undo commit, or leave them as they
are.

That's what I mean by changing the behaviour in ways that a simple
"what's the latest version published in $pocket" may start reporting as
out of date.

Thanks,

James

Martin Pool (mbp) wrote :

It seems to me that taking this approach in this context at least
won't make things any worse with regard to -proposed.

Jelmer Vernooij (jelmer) wrote :

I've pushed a hackish implementation of a 'bzr check-udd-status' command to lp:~jelmer/bzr-builddeb/check-udd-status.

E.g.:

gwenhwyvar:~% bzr check-udd-status gnumeric
UDD branch is out of date (1.9.9-1ubuntu1 < 1.10.14-1ubuntu2)

It's pretty slow to do this using the API though:
BZR_PLUGINS_AT=builddeb@`pwd` BZR_PDB=1 bzr check-udd-status gnumeric 0.47s user 0.09s system 5% cpu 9.805 total

This would make fetching a UDD branch quite a bit slower. We can presumably avoid opening the actual branch like this command does, but that would still mean something like an extra 5 second overhead for fetching a branch.

John A Meinel (jameinel) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 5/20/2011 6:26 PM, Jelmer Vernooij wrote:
> I've pushed a hackish implementation of a 'bzr check-udd-status' command
> to lp:~jelmer/bzr-builddeb/check-udd-status.
>
> E.g.:
>
> gwenhwyvar:~% bzr check-udd-status gnumeric
> UDD branch is out of date (1.9.9-1ubuntu1 < 1.10.14-1ubuntu2)
>
> It's pretty slow to do this using the API though:
> BZR_PLUGINS_AT=builddeb@`pwd` BZR_PDB=1 bzr check-udd-status gnumeric 0.47s user 0.09s system 5% cpu 9.805 total
>
> This would make fetching a UDD branch quite a bit slower. We can
> presumably avoid opening the actual branch like this command does, but
> that would still mean something like an extra 5 second overhead for
> fetching a branch.
>

Getting the branch tip over http might be a fair amount faster than
doing so over bzr+ssh if you aren't going to fetch the branch. (Since
you avoid the ssh handshake.)

I don't know where the slow part of your code is, though. It certainly
could be bits like "Launchpad requires me to download all of the history
for a package to determine the most recent version". Etc.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk3X1WIACgkQJdeBCYSNAAMHzgCfSHigu7Nk65n7wRiDibEDd0iU
pyAAn284nBeyY58h7Fu3loIaKpL8mkZ1
=aNdb
-----END PGP SIGNATURE-----

Jelmer Vernooij (jelmer) wrote :

On Sat, 2011-05-21 at 15:08 +0000, John A Meinel wrote:
> On 5/20/2011 6:26 PM, Jelmer Vernooij wrote:
> > I've pushed a hackish implementation of a 'bzr check-udd-status' command
> > to lp:~jelmer/bzr-builddeb/check-udd-status.
> >
> > E.g.:
> >
> > gwenhwyvar:~% bzr check-udd-status gnumeric
> > UDD branch is out of date (1.9.9-1ubuntu1 < 1.10.14-1ubuntu2)
> >
> > It's pretty slow to do this using the API though:
> > BZR_PLUGINS_AT=builddeb@`pwd` BZR_PDB=1 bzr check-udd-status gnumeric 0.47s user 0.09s system 5% cpu 9.805 total
> >
> > This would make fetching a UDD branch quite a bit slower. We can
> > presumably avoid opening the actual branch like this command does, but
> > that would still mean something like an extra 5 second overhead for
> > fetching a branch.
> Getting the branch tip over http might be a fair amount faster than
> doing so over bzr+ssh if you aren't going to fetch the branch. (Since
> you avoid the ssh handshake.)

> I don't know where the slow part of your code is, though. It certainly
> could be bits like "Launchpad requires me to download all of the history
> for a package to determine the most recent version". Etc.
I'm currently opening the branch over http, and am only looking at the
tags - not at any of the actual history.

It seems like most of the overhead is in the roundtrips to Launchpad for
the API. Maxb did some work yesterday on reducing that number.

Cheers,

Jelmer

Max Bowsher (maxb) wrote :

The conclusion of that work was that it is entirely possible to hit the Launchpad web service manually, rather than using launchpadlib, so that you have absolute control of the number of roundtrips.

The caveats involved are:

1) Because of bug 714038, it is necessary to submit an OAuth Authorization header even though it should not be. However, a simple fixed header value works fine:

Authorization: OAuth realm="", oauth_consumer_key="something", oauth_signature_method="PLAINTEXT", oauth_version="1.0", oauth_token="", oauth_signature="%26"

2) It is necessary to be aware of lazr.restful's data type marshalling - basically, data is represented as if it were fragments of JSON, which means string parameters need quotes around them.

3) If calling a method that returns a collection that might possibly ever have more than a few items, you need to care about collection pagination.

With that in mind, we can get it down to just 1 roundtrip for looking up the currently published source version in a named distroseries, and 2 roundtrips for the same in the implicit development focus distroseries case. (Or we could just call Archive#getPublishedSources without a distroseries limit, and pick the latest, to keep that down to 1 roundtrip too)

Martin Pool (mbp) wrote :

> 2) It is necessary to be aware of lazr.restful's data type marshalling -
> basically, data is represented as if it were fragments of JSON, which
> means string parameters need quotes around them.

It seems like that ought to be in the api docs. I wonder if it is.

Martin Pool (mbp) wrote :

well, I added this to https://help.launchpad.net/API/Hacking because
it has surprised me before too

John A Meinel (jameinel) on 2011-07-11
Changed in udd:
assignee: canonical-bazaar (canonical-bazaar) → John A Meinel (jameinel)
John A Meinel (jameinel) on 2011-07-13
Changed in bzr:
status: Confirmed → In Progress
assignee: nobody → John A Meinel (jameinel)
John A Meinel (jameinel) wrote :

Needs polish, but it will be in the bzr-2.5 series.

Changed in bzr:
status: In Progress → Fix Released
milestone: none → 2.5b1
John A Meinel (jameinel) wrote :

We chose to do this with a freshness RPC, rather than doing a MotD sort of arrangement. So it doesn't require changes in Launchpad.

Changed in udd:
status: Triaged → Fix Released
Changed in launchpad:
status: Triaged → Invalid
Martin Pool (mbp) wrote :

after discussion in the udd meeting, i think we should backport this to lucid at least (2.1) and it will be reasonable to put into srus

John A Meinel (jameinel) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/27/2011 2:14 PM, Martin Pool wrote:
> after discussion in the udd meeting, i think we should backport this
> to lucid at least (2.1) and it will be reasonable to put into srus
>

I believe I started the code against bzr-2.2 because there was some
specific difference between 2.1 and 2.2 (maybe something with the test
suite?).

But sure, it does make sense to backport something like this to the
latest LTS.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk479i0ACgkQJdeBCYSNAAMBgACgvQE4aq/XfPY1hwCWm1PzT1zx
TX0AnR9OP2jLXlzIcNI7eNYHIZpKDJkC
=F8uM
-----END PGP SIGNATURE-----

Martin Pool (mbp) wrote :

I think this needs a SRU into maverick of bzr 2.2.5.

Jelmer Vernooij (jelmer) on 2011-09-15
Changed in bzr (Ubuntu):
status: New → Fix Released
importance: Undecided → High
Martin Pitt (pitti) wrote :

Is there really much point in fixing this in maverick now? It has lived most of its live with this bug, and will go EOL in two months.

Martin Pool (mbp) wrote :

I think we might as well drop the maverick task.

Martin Pitt (pitti) on 2012-02-17
Changed in bzr (Ubuntu Maverick):
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers