cibuild support for bzr branches

[Jelmer Vernooij]

What would it take for launchpad to support ci builds for bzr branches?

I'm possibly interested in contributing some changes, but it'd be great to have some pointers of where to change things exactly. Thanks!

[Jürgen Gmach (jugmac00)]

I can't tell you what it would take for bzr branches, but you could have a look at how this works for git - it all starts with an event created when a ref is updated:
https://git.launchpad.net/launchpad/tree/lib/lp/code/subscribers/git.py

Having a look at https://git.launchpad.net/launchpad/tree/lib/lp/code/model/cibuild.py we do not abstract the vcs, but that should be doable.

We def. need to go down the stack through launchpad-buildd and lpcraft(the CI runner). launchpad-buildd's https://git.launchpad.net/launchpad-buildd/tree/lpbuildd/ci.py is designed to work with git only - but probably also could be refactored and extended.

lpcraft ( https://lpcraft.readthedocs.io/en/latest/ ) should be vcs agnostic if I remember correctly.

So this seems to be doable, with quite some effort, and especially some coordinated one (align launchpad and launchpad-buildd changes).

The most important thing would be whether this is something we want to have and maintain in the long term. That is something we need to discuss with the team, especially with Colin and probably our "manager". Colin is off for probably another week. I will discuss this once he's back.

[Jelmer Vernooij (jelmer)]

> We def. need to go down the stack through launchpad-buildd and lpcraft(the CI runner). launchpad- buildd's https://git.launchpad.net/launchpad-buildd/tree/lpbuildd/ci.py is designed to work with git only - but probably also could be refactored and extended.

Are you sure? it seems to at least take a branch argument.

[Colin Watson (cjwatson)]

I don't think launchpad-buildd needs any work, indeed; we reused existing code there which handles both bzr and git. (Of course nobody has ever tested CI jobs with bzr there, so there may be bugs.)

Both the `RevisionStatus` and `CIBuild` tables would need `branch` columns, and the `NOT NULL` constraint on their `git_repository` columns would need to turn into "must have either branch or git_repository" constraints; something would also need to be done with the `commit_sha1` column, perhaps an alternative involving bzr revision IDs or an FK to the `Revision` table, and a number of indexes will need to be added or updated.

The corresponding `RevisionStatus` and `CIBuild` Python models would need similar updates, and `CIBuild.getConfiguration` method would need to be taught the alternative of fetching `.launchpad.yaml` from a bzr branch (not too hard, as `Branch.getBlob` exists nowadays). `CIBuildBehaviour` would need to be taught about the bzr case and how to pass the correct arguments when dispatching a build. Various bzr views would need to do the same sorts of things that git views do to show revision status information (grep for `show_status_reports`). There'd need to be a subscriber hooked into the branch scanner to request CI builds on push, probably in `lp.codehosting.scanner.bzrsync`.

I may be forgetting a few pieces, but I think that's the essence of it. The Canonical Launchpad team won't be adding this feature ourselves, but none of this is so complicated or invasive that I'd be concerned about us *maintaining* it, so if you're interested in contributing support for it then I think we could accept it.

I'd recommend doing the bits related to `RevisionStatus` first (including equivalents of `GitRepository.newStatusReport` and `GitRepository.getStatusReports`), as that's the order in which we added this for git, and of course changes in smaller chunks are easier to review. It should be possible to demonstrate the ability to POST revision statuses externally, and then add in the feature of having LP generate them itself.