[needs packaging] juju2 beta is not in xenial

[Curtis Hovey]

Juju 2.0 will be in xenial. We want to place juju 2.0-beta2 (or possibly beta3) in xenial for Ubuntu users to test.

This branch introduces the juju2 package.

    lp:~juju-qa/ubuntu/xenial/juju2/xenial-2.0-beta2

The juju team propose a new juju2 packages that is co-installable with
the juju-core package. This allows users to have juju 2.x and 1.x
installed on the xenial host. Users can maintain their 1.x envs
while creating new Juju 2.x deployments.

This branch is based on
    lp:~sinzui/ubuntu/xenial/juju-core/xenial-1.25.4
Which is stalled. We are waiting for a 1.25.5 released. We want to
use this as the base because it minimises the changes in the debian
dir.

  * New upstream release (Lp: 1560315).
  * d/control:
    - Renamed the package juju-core to juju2.
    - Dropped unneeded juju-local* and juju packages.
  * d/rules:
    - Renamed the package to juju2.
    - Updated rules to 2.x.
    - Additional path/name changes to support co-installability with
      the 1.x juju-core package.
  * d/copyright: Updated per changes to embedded dependencies.
  * d/tests/*
    - Updated tests to use juju2 command line.
    - Replaced local-provider tests with lxd tests.

[Devel Fix]

[sinzui] Prepare source package and diffs: DONE
[stokachu] Review package: DONE
[stokachu] Upload to the development release (xenial-proposed): DONE

[sinzui] Upstream QA test against xenial: TODO
[sinzui] Test streams with xenial-proposed: TODO

[Curtis Hovey (sinzui)]

The attaches diff shows the changes to the juju embedded dependencies.

[Curtis Hovey (sinzui)]

The attached diff shows the changes to debian/ from the 1.25.4 branch (lp:~sinzui/ubuntu/xenial/juju-core/xenial-1.25.4). The 1.25.4 branch is stalled. We are waiting on a 1.25.5 to fix a regressions. We do not expect any other changes to the debian dir other than the changelog.

[Curtis Hovey (sinzui)]

Attached is a gz diff of the changes in src/ from juju 1.25.4 to 2.0-beta2. Sorry. This is hell to read.

[Adam Stokes (adam-stokes)]

For completeness this is the lintian output in all its glory:

E: juju2 changes: bad-distribution-in-changes-file xenial-amd64
N:
N: You've specified an unknown target distribution for your upload in the
N: debian/changelog file. It is possible that you are uploading for a
N: different distribution than the one Lintian is checking for. In that
N: case, passing --profile $VENDOR may fix this warning.
N:
N: Note that the distributions non-free and contrib are no longer valid.
N: You'll have to use distribution unstable and Section: non-free/xxx or
N: Section: contrib/xxx instead.
N:
N: Refer to Debian Policy Manual section 5.6.14 (Distribution) for details.
N:
N: Severity: important, Certainty: certain
N:
N: Check: changes-file, Type: changes
N:
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju ang and
N:
N: Lintian found a spelling error in the given binary. Lintian has a list
N: of common misspellings that it looks for. It does not have a dictionary
N: like a spelling checker does.
N:
N: If the string containing the spelling error is translated with the help
N: of gettext or a similar tool, please fix the error in the translations
N: as well as the English text to avoid making the translations fuzzy. With
N: gettext, for example, this means you should also fix the spelling
N: mistake in the corresponding msgids in the *.po files.
N:
N: You can often find the word in the source code by running:
N:
N: grep -rw <word> <source-tree>
N:
N: This tag may produce false positives for words that contain non-ASCII
N: characters due to limitations in strings.
N:
N: Severity: minor, Certainty: wild-guess
N:
N: Check: binaries, Type: binary, udeb
N:
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju Succesfully Successfully
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju enviroment environment
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju specifed specified
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju transfered transferred
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju reenabled re-enabled
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju intial initial
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju reenable re-enable
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju ommitted omitted
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju precendence precedence
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju allows to allows one to
E: juju2: embedded-library usr/lib/juju-2.0-beta2/bin/juju: libyaml
N:
N: The given ELF object appears to have been statically linked to a
N: library. Doing this is strongly discouraged due to the extra work needed
N: by the security team to fix all the extra embedded copies or trigger the
N: package rebuilds, as appropriate.
N:
N: If the package uses a modified version of the given library it is highly
N: recommended to coordinate with the library's maintainer to include the
N: changes on th...

[Curtis Hovey (sinzui)]

Ouch
> For completeness this is the lintian output in all its glory:

> E: juju2 changes: bad-distribution-in-changes-file xenial-amd64

I certainly could have put "xenial-amd64" in the changelog, but i do not see in debian. my find also fails

>E: juju2: embedded-library usr/lib/juju-2.0-beta2/bin/juju: libyaml

I don't see juju linking to libyaml. It do see it linking to two versions of the go port though. I know Core want to drop the v1 of yaml. I have reviewed the output a a build and do not see libyaml being linked.

[Adam Stokes (adam-stokes)]

With my latest changes here is the lintian report:

I: juju2 source: wildcard-matches-nothing-in-dep5-copyright src/github.com/chai2010/gettext-go (paragraph at line 58)
N:
N: The wildcard that was specified matches no file in the source tree. This
N: either indicates that you should fix the wildcard so that it matches the
N: intended file or that you can remove the wildcard. Notice that in
N: contrast to shell globs, the "*" (star or asterisk) matches slashes and
N: leading dots.
N:
N: Refer to
N: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
N: details.
N:
N: Severity: minor, Certainty: possible
N:
N: Check: source-copyright, Type: source
N:
I: juju2 source: wildcard-matches-nothing-in-dep5-copyright src/github.com/chai2010/gettext-go (paragraph at line 65)
I: juju2 source: wildcard-matches-nothing-in-dep5-copyright src/golang.org/x/net/html/charset/testdata/* (paragraph at line 455)
W: juju2 source: dep5-copyright-license-name-not-unique (paragraph at line 1328)
N:
N: This paragraph defines an already defined license.
N:
N: According to the specification, short license names are required to be
N: unique within a single copyright file.
N:
N: Refer to
N: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
N: details.
N:
N: Severity: normal, Certainty: possible
N:
N: Check: source-copyright, Type: source
N:
W: juju2 source: dep5-copyright-license-name-not-unique (paragraph at line 1615)
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/LICENSE
N:
N: The source tree contains a file which was not matched by any of the
N: Files paragraphs in debian/copyright. Either adjust existing wildcards
N: to match that file or add a new Files paragraph.
N:
N: Refer to
N: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ for
N: details.
N:
N: Severity: normal, Certainty: possible
N:
N: Check: source-copyright, Type: source
N:
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/README.md
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/Makefile
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/hello.go
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/hi/hi.go
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/local.zip
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/local/default/LC_MESSAGES/hello.mo
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/local/default/LC_MESSAGES/hello.po
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/local/default/LC_RESOURCE/hello/favicon.ico
W: juju2 source: file-without-copyright-information src/github.com/chai2010/gettext-go/examples/local/default/LC_RESOURCE/hello/poems.txt
W: juju2 source: file-without-copyright-information src/github.com/chai2010/g...

[Adam Stokes (adam-stokes)]

Ok I've gotten it down to some minor packaging issues:

I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju ang and
N:
N: Lintian found a spelling error in the given binary. Lintian has a list
N: of common misspellings that it looks for. It does not have a dictionary
N: like a spelling checker does.
N:
N: If the string containing the spelling error is translated with the help
N: of gettext or a similar tool, please fix the error in the translations
N: as well as the English text to avoid making the translations fuzzy. With
N: gettext, for example, this means you should also fix the spelling
N: mistake in the corresponding msgids in the *.po files.
N:
N: You can often find the word in the source code by running:
N:
N: grep -rw <word> <source-tree>
N:
N: This tag may produce false positives for words that contain non-ASCII
N: characters due to limitations in strings.
N:
N: Severity: minor, Certainty: wild-guess
N:
N: Check: binaries, Type: binary, udeb
N:
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju Succesfully Successfully
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju enviroment environment
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju specifed specified
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju transfered transferred
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju reenabled re-enabled
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju intial initial
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju reenable re-enable
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju ommitted omitted
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju precendence precedence
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju allows to allows one to
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-metadata ang and
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-metadata enviroment environment
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-metadata specifed specified
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-metadata arbitary arbitrary
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-metadata precendence precedence
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-restore ang and
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-restore specifed specified
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-upgrade-mongo ang and
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/juju-upgrade-mongo specifed specified
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/jujud ang and
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/jujud Succesfully Successfully
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/jujud enviroment environment
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/jujud specifed specified
I: juju2: spelling-error-in-binary usr/lib/juju-2.0-beta2/bin/jujud addreses address...

[Martin Pitt (pitti)]

FTR, I did an initial source NEW review of juju2. In the initial MIR bug 1267393 (two years ago) the deal was to *reduce* the embedded code copies and replace them with golang-google-api-dev & co. build dependencies. juju-core now does have a handful of those, but still an awful lot of embedded copies which are available in the archive (such as golang-github-bmizerany-pat-dev).

But the current juju2 package in NEW goes back to square 1: It *solely* uses embedded code copies (including packages which are very active and well maintained in Ubuntu such as lxd!), and there's now about 50% more modules bundled.

Please rework the package to use archive dependencies as much as possible. Thanks!

[Jamie Strandboge (jdstrand)]

The agreement between the juju core team, MIR, foundations and security teams for 16.04 was that the juju would:

 1. test out use of -buildmode=shared/-linkshared for 16.04, but not be committed to using them since the feature is new
 2. the number of embedded copies that juju uses as part of the 16.04 archive release build would be significantly reduced by using 16.04 golang -dev packages (bug #1508120). It is understood this number will be non-zero (ie, juju will need to build with a few)
 3. juju is allowed to ship other embedded copies of software that aren't used as part of the build to facilitate out-of-archive rebuilds on older releases

There appears to be no progress on bug #1508120, at least according to the bug. What is the status of this?

What is the status of '1'?

Of the packages that juju2 is embedding, how many are used as part of the 16.04 archive build?

[Martin Packman (gz)]

Hm, looks like the adventures with reworking the packaging yesterday (which I didn't follow the reasoning behind, but ended up basing the beta2 packaging branch on an existing xnox branch) lost the Build-Depends we had on archive golang packages.

What we had:

<http://bazaar.launchpad.net/~sinzui/ubuntu/xenial/juju-core/xenial-1.25.4/view/head:/debian/control>

Missing here:

<http://bazaar.launchpad.net/~juju-qa/ubuntu/xenial/juju2/xenial-2.0-beta2/view/head:/debian/control>

So, they should be added back in.

That said, the juju qa team hasn't spent any time this cycle on adventures with golang packaging. What time we have had on distro tasks has been sucked up basically packaging mongo 3 (and mongo 2.6 for migration purposes).

[Nicholas Skaggs (nskaggs)]

I've added 12 dependencies for the go modules used by juju as depends in in the control file. This is a step back in the proper direction. There are a couple more depends we could add based upon what's in the archive that is currently breaking the build (one of them is lxd). Beyond that, we'll need to start getting packages created for the other dependencies we need. I trust these changes make it more palatable towards entering into universe.

[Adam Stokes (adam-stokes)]

We are using juju-core package name now so there is no need for this packaging request :)