Comment 68 for bug 1267393

Yesterday slangasek and I met with the juju team regarding the path forward for transitioning to shared libraries and Alexis will be commenting on that later.

Unrelated to that, juju is using many embedded code copies and juju-core will need to be adjusted to build with many of these broken out into golang-*-dev packages (see https://wiki.ubuntu.com/MIRTeam for details, and use dh-make-golang to make this easy). Before I give the list, a couple of things:
 * it is acknowledged that the Go community has traditionally embedded sources in their trees and then managed imports themselves. This works fine for the app model where bundling is the norm (eg, for Ubuntu Touch clicks and Ubuntu Core snaps). However, for the reasons outlined in the MIRteam wiki, this does not work well with the Linux distribution model and applying for main inclusion in Ubuntu by definition means working within the distribution model. Also, the Go community has acknowledged the benefits of shared libraries with regard to code reuse, project management, the distribution model, etc, which is why Go 1.5 has shared library capability (that doesn't mean the app bundling model doesn't still have its place)
 * it is also acknowledged that juju is different in some ways to other software delivered via the archive in that its current packaging of client, server, agents, etc is for multiple platforms, not just Ubuntu. However, juju is being delivered to Ubuntu juju clients via apt and therefore for those clients there isn't a technical reason why those embedded libraries can't be delivered as separate packages (in other words, it's fine if juju-core has embedded sources in the orig.tar.gz if that helps the juju team, it is just that (many of) those embedded sources should not be used during archive builds (where instead juju-core should use Build-Depends on golang-*-dev packages)). This is all standard Debian and Ubuntu archive practice and that is why the Debian Go team has been working so hard on dh-golang, Built-Using, etc
 * it should be acknowledged by go developers seeking main inclusion that the MIR, foundations and security teams have already made many concessions for golang and these changes to the process alone will result in increased maintenance effort for officially supported packages (but at a level we believe can be supported, unlike if we ignored embedded sources/shared libraries)
 * creating golang-*-dev packages for embedded sources doesn't mean the juju team can't still control the code, it is just that the lib will live in a different place. Eg, suppose that github.com/joyent/* is pulled out into golang-joyent-dev. A MIR would be required for golang-joyent-dev and the juju team would be the team committed to maintaining the package and fixing those bugs. The PPU acls would be adjusted to allow the juju team to upload this package, just like with juju-core. The SRU process would apply to it, etc. (Ie, nothing fundamentally changes wrt control if the juju team wants to maintain that control (ie, perhaps they wouldn't mind relinquishing control of some of them))
 * all the embedded packages do not necessarily need to be pulled out. Eg, github.com/juju and gopkg.in/juju seem to be internal to juju and if nothing in the archive would ever need to use them, there is no reason to break them out
 * juju is currently suffering from the problems that can happen with the bundling model in that it is using different embedded sources for the same functionality. Eg, code.google.com/p/go.crypto and golang.org/x/crypto. Picking one and breaking it out into a single -dev package will ease the maintenance burden

With that said, here is a list to start the conversation for *Ubuntu archive builds* and using embedded sources (again, if it helps to leave the sources in the orig.tar.gz for other reasons, fine):

* embedded that seem obviously ok to leave alone and use during the build:
 * github.com/juju/* (do pull out anything that's reused in other go sources though)
 * gopkg.in/juju/* (same here)

* embedded that seems clear should be cleaned up/pulled out/use existing archive -dev package:
 * code.google.com/p/go.crypto (use golang-go.crypto-dev, why this and golang.org/x/crypto? pick one)
 * golang.org/x/crypto (why this and code.google.com/p/go.crypto? pick one)
 * golang.org/x/net (use golang-golang-x-net-dev)
 * gopkg.in/check.v1 (use golang-check.v1-dev)
 * gopkg.in/mgo.v2 (use golang-gopkg-mgo.v2-dev)
 * gopkg.in/yaml.v1 (use golang-goyaml-dev, but consider golang-yaml.v2-dev cause that is what snappy uses and the MIR process advocates for one package per functionality as much as possible)

* embedded that may need other Canonical upstream involvement to create golang-*-dev packages (note, LXD is applying for MIR in 15.10 so this golxc might fall under that work):
 * launchpad.net/golxc (seems our lxc packages should be adjusted to provide golang-golxc-dev?)
 * launchpad.net/gomaasapi (seems like our maas packages should be adjusted to provide golang-gomaasapi-dev?)

* embedded sources where is is unclear if it is juju-specific or something that should be broken out
 * bitbucket.org/kardianos/osext (seems like something useful to others? ie, break out?)
 * bitbucket.org/kardianos/service (seems like something useful to others? ie, break out?)
 * code.google.com/p/winsvc (seems like something useful to others? ie, break out?)
 * github.com/bmizerany/pat (??)
 * github.com/joyent/* (juju-specific?)
 * gopkg.in/natefinch/lumberjack.v2 (??)
 * gopkg.in/natefinch/npipe.v2 (seems like something useful to others? ie, break out?)
 * launchpad.net/gnuflag (seems like something useful to others? ie, break out?)
 * launchpad.net/goamz/* (juju-specific? (amazon))
 * launchpad.net/goose/* (juju-specific? (openstack))
 * launchpad.net/gwacl/* (seems like something useful to others? ie, break out?)
 * launchpad.net/tomb (??)

Questions:
* juju team: can you comment on the package breakdown? For items requiring further discussion, it might be worthwhile understanding how often you are updating the embedded package (useful for the SRU question, below)
* SRU team: juju-core already has a release exception. For packages that are being broken out that were formerly part of the juju-core package and that the juju team will now maintain, can those just be given a release exception?
* Ubuntu Archive team: juju-core will likely need a PPU for members of the juju team when it goes to main. Can we extend the acl to include the packages that are being broken out that they are going to maintain?
* MIR team: for the packages that are being broken out, I propose that they don't get extended MIR review, but rather simply the packaging review to make sure they are following the Go standards as outlined in the MIRteam document
* juju team (/security team): the juju team has said that they would like coordination of security updates for juju-core and golang-*-dev packages for which they maintain. I propose the security team maintains a list of packages and when we triage a CVE against a package in that list, we file a bug for the juju team to fix, and sponsor their uploads (like for other Canonical upstreams). juju team-- does that address your concerns?