Need way to list and observe branch tips

Bug #781600 reported by Gustavo Niemeyer
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Critical
Benji York

Bug Description

For the Ensemble work we've been debating, we need a way to list (location, tip revision id, (official_series, official_series)) for all branches of a distribution (principia, in our case).

Since this list will hopefully get huge over time, we'll also need a way to ask for the changes since a given date, in which case we'd only get results for those branches that are either new or have had changes to their tip revision id since the provided timestamp.

Please note that we don't need/want *all the data* for a branch in that list, since it's going to be quite a lot of unnecessary information being transferred over the wire.

For reference only, in a conversation with Jono over UDS, he's come up with this sketched query which would do something resembling what we'd need:

SELECT unique_name, last_scanned_id FROM Branch JOIN DistroSeries ON
Branch.distroseries = DistroSeries.id JOIN Distribution ON
DistroSeries.distribution = Distribution.id WHERE Distribution.name =
'ubuntu' AND branch.last_scanned > '2011-05-01';

Related branches

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 781600] [NEW] Need way to list and observe branch tips

all branches, or all the branches aliased (the 'official branches')?

Curtis Hovey (sinzui)
Changed in launchpad:
status: New → Triaged
importance: Undecided → Low
tags: added: branches
Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

All branches. People can publish any branches they want as formulas. The official ones are just a selection of those.

tags: added: principia
Revision history for this message
Francis J. Lacoste (flacoste) wrote :

Although adding a method just for this use case would work. I'd rather we export a getBranches() method which returns the full branch information and which takes parameters to query the results. That would useful in more use cases and should still work for you (at the expense of returning a little more information than you stricly require).

Changed in launchpad:
importance: Low → Critical
tags: added: escalated
Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

Note that it's not just a bit more, but _a lot_ more information. All we need is really just a couple of strings (location, revid). Compare that with the 40+ (!) fields of a branch, including the _whiteboard_. Then, imagine iterating over a potentially long list of these. I'm generally against early optimization, but this is a lot of overhead.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 781600] Re: Need way to list and observe branch tips

On Sat, Jun 11, 2011 at 12:56 PM, Gustavo Niemeyer <email address hidden> wrote:
> Note that it's not just a bit more, but _a lot_ more information.  All
> we need is really just a couple of strings (location, revid).  Compare
> that with the 40+ (!) fields of a branch, including the _whiteboard_.
> Then, imagine iterating over a potentially long list of these.  I'm
> generally against early optimization, but this is a lot of overhead.

So the current Launchpad API is very much about exporting /objects/.
This is why its so flexible, and why its so slow. AIUI its baked into
lazr.restful at a fairly deep level.

We're expecting that you will want to examine between 1K and 20K
objects; this is probably outside what we can reliably do in 1 second
even for the minimal data - so will need batching etc.

However, we can do a view returning json of the results of your query,
with batching, pretty easily, even if the API framework would be in
our way.

So I'm in favour of us having an *unstable* (that is, devel only, no
long term support guarantee) API that precisely fits your needs.
Should we need to change it in future we'd coordinate that with you :
the lack of long term support guarantee says that we're not supporting
this for shipping in API clients on the next Ubuntu LTS (if you need
that, you should say so now). Generally for webapps and
backend-in-the-datacentre unstable APIs are fine - we don't change
them willy-nilly.

-Rob

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

Ensemble will ship in the next LTS, but this is not going to be _in_ the LTS. As you say, it's something for the data center, so we should be fine.

description: updated
Benji York (benji)
Changed in launchpad:
assignee: nobody → Benji York (benji)
status: Triaged → In Progress
Benji York (benji)
description: updated
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
Revision history for this message
William Grant (wgrant) wrote :

Works fine with the date restriction. And, impressively, the unrestricted variant renders in under 5s most of the time on qastaging, into a glorious 30MB JSON document containing every Ubuntu branch.

tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers