don't publish packages to overlay ppa until the proposed migration succeeded

[Michael Zanetti]

When a ticket is published, the packages are immediately copied to the overlay ppas. However, the code is not merged until the packages migrate through devel-proposed.

This leaves our developer machines out of sync for quite some time, assuming that the tests pass. However, due to flakiness on britney, actual test failures etc it frequently happens, that our developer machines stay out of sync with the code we work on for multiple days.

It would be great if there wasn't such a huge skew between the packages appearing in the overlay PPA and the code being merged to trunk.

[Alberto Mardegan (mardy)]

Just my 2 cents here.

I believe that part of the problem is that we have only one branch (trunk) apparently tracking different releases. To me, "trunk" is the tip of development, and as such should track the newest Ubuntu release under development, that currently is "zesty". So, indeed it is correct that changes are not merged into trunk until they land of zesty; what is probably not correct, is that packages get immediately uploaded to the overlay PPAs while this fact is not reflected in any source code branch.

My suggestion (where #1 is the more immediate one, #2 would be an improvement over #1):

1) Keep everything as is, but delay the upload to the overlay PPAs and sync it with the merging of code into trunk.

2) Define a naming scheme for branches tracking the overlay releases (like "vivid-overlay", "xenial-overlay", maybe); then, when landing a silo, for each target series check if the corresponding branch exists in the project and, if found, sync the landing to the overlay and the code merge, independently from all other series. For those series where a corresponding branch is not found, delay the landing according to #1.

Not sure how much sense this makes, and whether it's all needed. Just brainstorming :-)

[Robert Bruce Park (robru)]

#2 is fundamentally unworkable because the packages being uploaded to the overlay PPA are not tracked in any VCS at all. The way overlay packages are created, at least in dual/triple tickets, is that the devel series is merged together from MPs, but the stable series just has the directory copied and then debian/changelog mangled to reflect the older series, and then the source package is uploaded to the PPA. So eg there is no associated MP with that, there's no branch it can copy to/from. The class that defines the behavior of the overlay packages purposely has no knowledge of any VCS whatsoever.

From my perspective there are two possible solutions:

A. If you want to continue using dual tickets, we can delay the upload to the overlay PPA to happen when zesty-proposed migrates and branches are being merged.

B. You can just stop using dual tickets, eg, do one ticket for your zesty release and then after that, do a second ticket for your overlay PPA release. This is obviously a lot more work, having twice as many tickets & merges necessary to get your releases out, but it at least correctly avoids the issue of uploading things to overlay prematurely, and is possible to do today, without any changes from me.

I would like to get Steve's input on this as IIRC I proposed (A) some years ago and he veto'd it but I can't recall why.

[Steve Langasek (vorlon)]

> A. If you want to continue using dual tickets, we can delay the
> upload to the overlay PPA to happen when zesty-proposed migrates
> and branches are being merged.

> I would like to get Steve's input on this as IIRC I proposed (A)
> some years ago and he veto'd it but I can't recall why.

I don't recall now a specific suggestion around this, or my objections to it, sorry. Possibly related to rollback scenarios?

But the right answer is still to stop using dual landing tickets, as these are fundamentally flawed.

[Robert Bruce Park (robru)]

Do you have any suggestions what a good workflow would look like without being overly burdensome? The dual tickets were designed as a convenience but if it's causing problems then we should scrap it and build something better.

[Steve Langasek (vorlon)]

On Thu, Dec 15, 2016 at 09:13:04AM -0000, Robert Bruce Park wrote:
> Do you have any suggestions what a good workflow would look like without
> being overly burdensome?

Branch per series; two MPs, two tickets.

> The dual tickets were designed as a convenience but if it's causing
> problems then we should scrap it and build something better.

There are fundamental problems with dual landings that will never go away.
Even when everyone is following the process to the letter, a dual landing
can always be stalled because of in-progress transitions in the devel
series, and some of these can take days or weeks to unpick. I don't think
the savings from only having to handle a single branch outweighs the costs
of having landings for a stable series wedged due to devel-proposed.