de-vendorize golang-go.crypto from juju-core

[Mathieu Trudel-Lapierre]

[Impact]
Go software using crypto modules. Juju-core was accepted in the archive with a vendorized version of golang-go.crypto at the last minute, but it should be removed and the archive version used instead.

[Test case]
- building Juju -
build juju-core, make sure it uses golang-golang-x-crypto-dev.

- rebuild tests for reverse dependencies -
rebuild r-deps for golang-go.crypto.

[Regression Potential]
New failure modes in building reverse-dependencies of crypto, or to build/run juju would constitute a regression of this update.

----

juju-core currently ships a copy of golang-go.crypto with itself. It shouldn't, and should instead use the copy of golang-go.crypto from the archive by Build-Depending on golang-golang-x-crypto.

This requires a newer snapshot of golang-go.crypto as juju-core or golang-go.net require the acme package from crypto, which is not properly exported in golang-go.crypto 1:0.0~git20160824.0.351dc6a-1ubuntu1.

[Mathieu Trudel-Lapierre (cyphermox)]

There's now https://launchpad.net/ubuntu/+source/golang-go.crypto/1:0.0~git20161012.0.5f31782-1ubuntu1 in zesty.

[Steve Langasek (vorlon)]

Hello Mathieu, or anyone else affected,

Accepted golang-go.crypto into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/golang-go.crypto/1:0.0~git20161012.0.5f31782-1ubuntu0.16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Mathieu, or anyone else affected,

Accepted golang-go.crypto into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/golang-go.crypto/1:0.0~git20161012.0.5f31782-1ubuntu0.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Mathieu Trudel-Lapierre (cyphermox)]

verification-failed: there are various issues with de-vendorizing crypto in yakkety, including the need to rebuild half of the golang world; it was decided to land juju-core without the de-vendorizing in yakkety.

[Mathieu Trudel-Lapierre (cyphermox)]

The same story applies to xenial.

[Adam Conrad (adconrad)]

"It was decided" by whom? If rebuilding "the world" due to a golang build-dep is a problem, then we have a massively poor security story here, and that needs sorting, not hand-waving past as "meh, too hard, guess we never change golang-crypto."

[Michael Hudson-Doyle (mwhudson)]

On 22 November 2016 at 08:40, Mathieu Trudel-Lapierre <email address hidden>
wrote:

> The same story applies to xenial.

Are you sure? No Go shared libraries in Xenial.

[Tyler Hicks (tyhicks)]

Adam, the decision was made between Mathieu, Steve Langasek, and myself.

My understanding was that juju-core was let into yakkety, shortly before the release, with a vendorized golang-go.crypto. That went against the MIR requirements and should not have happened. This SRU attempted to undo the vendorization but would have introduced quite a bit of regression risk in yakkety due to the need to bump the snapshot in the yakkety archive.

I was ok with allowing juju-core to continue to vendorize golang-go.crypto only in yakkety since yakkety is an intermediate release and this mistake was present in yakkety-release. Also, I requested that golang-go.crypto be de-vendorized in juju-core in zesty and continue to not be vendorized in xenial since it is an LTS and we'll be supporting it for quite some time.

[Mathieu Trudel-Lapierre (cyphermox)]

Michael, you're right, in xenial we do build with golang-go.crypto, which hasn't been migrated yet, but doesn't have the same golang rebuild issues as in yakkety.

[Mathieu Trudel-Lapierre (cyphermox)]

It was in an IRC conversation between me and Steve Langasek, after getting the Security team's approval (via tyhicks, also on IRC) that re-vendorizing golang-go.crypto was okay in yakkety, given that it's not a LTS release, and that there is at least another package (definitely snapd, and probably also lxd) that vendorizes everything.

My immediate concern isn't in the effort of rebuilding every reverse-dependency of golang-go.crypto, but of the high potential for regression involved in getting this new crypto and rebuilding everything against it, in the context of juju-core which is a project that is expected to change a lot. There WILL be other SRUs of juju-core in the future, and I don't know if other updates of golang packages may be required.

[Adam Conrad (adconrad)]

"Also, I requested that golang-go.crypto be de-vendorized in juju-core in zesty and continue to not be vendorized in xenial since it is an LTS and we'll be supporting it for quite some time."

Not convinced this is what's happened. Looking at a xenial build log, it certainly *builds* a local copy of crypto. And if, as this bug states, the crypto in yakkety was too old for juju, I don't see how the crypto in xenial would be magically okay, given that the juju versions are the same in both.

[Mathieu Trudel-Lapierre (cyphermox)]

golang-go.crypto is in xenial-proposed and the build certainly should have used it (and reports so, if only Built-Using can be trusted). If that's not the case, it's definitely a bug that needs to be fixed.

However, we need to consider this separately from yakkety. golang-go.crypto in yakkety should be removed, as it currently breaks other things in the archive (with this package, other golang packages will be uninstallable) and blocks other SRUs from being verified.

[Steve Langasek (vorlon)]

The yakkety-proposed upload has been removed; per the discussion in the bug I'm setting this back to v-needed for xenial.

[Ubuntu Foundations Team Bug Bot (crichton)]

The fix for this bug has been awaiting testing feedback in the -proposed repository for xenial for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

[Łukasz Zemczak (sil2100)]

As part of a recent change in the Stable Release Update verification policy we would like to inform that for a bug to be considered verified for a given release a verification-done-$RELEASE tag needs to be added to the bug where $RELEASE is the name of the series the package that was tested (e.g. verification-done-xenial). Please note that the global 'verification-done' tag can no longer be used for this purpose.

Thank you!

[Łukasz Zemczak (sil2100)]

The version of golang-go.crypto in the proposed pocket of Xenial that was purported to fix this bug report has been removed because the bugs that were to be fixed by the upload were not verified in a timely (105 days) fashion.