SRU: Shibboleth SPv3 for bionic

[Etienne Dysli Metref]

[Impact]

Bionic released with version 2 of the Shibboleth Service Provider (and its accompanying dependencies) and with OpenSSL 1.1. However, the SPv2 isn't compatible with OpenSSL 1.1, only 1.0 (and earlier), and was therefore shipped compiled against 1.0. This created a mix of OpenSSL and libcurl versions between the Apache module that the Shibboleth SP provides (mod_shib) and other modules, thus rendering mod_shib uninstallable alongside other modules (that depend on libcurl4) because of that conflict. Not being able to use mod_shib and mod_php with php-curl -- for example -- together greatly reduces the usefulness of the Shibboleth SPv2 in bionic, see LP#1776489. Version 3 of the Shibboleth SP is compatible with OpenSSL 1.1 and having it available for bionic would allow users to install it together with other Apache modules.

Moreover, the SPv2 suffers from a few security issues (LP#1636590) which have since been fixed upstream and v2 is no longer supported upstream (EOL, LP#1812401).

I propose to update the following source packages in bionic:
- shibboleth-sp [not in Bionic] to 3.0.4 (sync request for disco LP#1822055)
- opensaml [not in Bionic] to 3.0.1 (sync request for disco LP#1823325)
- xmltooling from 1.6.4-1ubuntu2.1 [Cosmic 3.0.2-1ubuntu1.1] to 3.0.4
- xml-security-c from 1.7.3-4ubuntu0.1 [Cosmic 2.0.1-1] to 2.0.2
- log4shib from 1.0.9-3 to 2.0.0
- shibboleth-resolver from 1.0.0-1build4 to 3.0.0

[Test Case]

# apt install apache2 libapache2-mod-shib2
[...]
# apt install libapache2-mod-php php-curl
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 php-curl : Depends: php7.2-curl but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

# apt install php7.2-curl
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 php7.2-curl : Depends: libcurl4 (>= 7.44.0) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

# apt install libcurl4
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libcurl3-gnutls libfcgi-bin libfcgi0ldbl liblog4shib1v5 libltdl7 libmemcached11 libodbc1 libssl1.0.0 libxerces-c3.2 libxml-security-c17v5 opensaml2-schemas shibboleth-sp2-common xmltooling-schemas
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  libapache2-mod-shib2 libcurl3 libsaml9 libshibsp-plugins libshibsp7 libxmltooling7 shibboleth-sp2-utils
The following NEW packages will be installed:
  libcurl4
0 upgraded, 1 newly installed, 7 to remove and 0 not upgraded.
Need to get 214 kB of archives.
After this operation, 18.7 MB disk space will be freed.
Do you want to continue? [Y/n] n
Abort.

[Regression Potential]
A new version can, of course, bring new bugs and security vulnerabilities. Catching up to SPv3 would at least give us an upstream-supported version. Shibboleth SP 3.0.4 and its dependencies are, as of this writing, all in Debian testing without any major bug.

[Etienne Dysli Metref (etienne-dysli-metref)]

Package log4shib 2.0.0.-2 from disco builds without changes on bionic, tested with `backportpackage --destination=bionic --source=disco --build --builder=cowbuilder --key=0x6965D453D81531AD log4shib`.

[Etienne Dysli Metref (etienne-dysli-metref)]

Package xml-security-c 2.0.2-3 from disco builds without changes on bionic, tested with `backportpackage --destination=bionic --source=disco --build --builder=cowbuilder --key=0x6965D453D81531AD xml-security-c`

[Etienne Dysli Metref (etienne-dysli-metref)]

Package xmltooling 3.0.4-1 from disco builds without changes on bionic, tested with `backportpackage --destination=bionic --source=disco --build --builder=cowbuilder --key=0x6965D453D81531AD xmltooling`.

[Etienne Dysli Metref (etienne-dysli-metref)]

Package opensaml 3.0.0-2 builds without changes on bionic, tested with `backportpackage --destination=bionic --source=disco --build --builder=cowbuilder --key=0x6965D453D81531AD opensaml`.

[Etienne Dysli Metref (etienne-dysli-metref)]

Package shibboleth-sp 3.0.3+dfsg1-1 from disco needs a revert from debhelper compat level 12 to 11. No other changes are required to build on bionic.

[Etienne Dysli Metref (etienne-dysli-metref)]

Would someone please review and sponsor this SRU?

[Etienne Dysli Metref (etienne-dysli-metref)]

The backported packages install and upgrade cleanly on bionic, tested with `piuparts -b /var/cache/pbuilder/base-bionic-amd64.tgz --distribution=bionic --keep-sources-list --arch=amd64 -D ubuntu --shell-on-error --single-changes-list log4shib_2.0.0-2~ubuntu18.04.1_amd64.changes xml-security-c_2.0.2-3~ubuntu18.04.1_amd64.changes xmltooling_3.0.4-1~ubuntu18.04.1_amd64.changes opensaml_3.0.0-2~ubuntu18.04.1_amd64.changes shibboleth-sp_3.0.3+dfsg1-1~ubuntu18.04.1_amd64.changes`.

[Robie Basak (racb)]

For each of the packages for which you're proposing to do a major version bump, please could you explain what the situation is for existing users of that package on Bionic? Could there be users of these packages who are happily using them but will find the rug pulled out under their feet due to functionality or behaviour changes? What's the situation with the reverse dependencies of the packages you're proposing to update?

[Robie Basak (racb)]

To be clear, what I'm asking for is an analysis, per source package, that explains the different use cases that could be being used currently in Bionic, and how they will be affected by this proposed update. The use cases need to include direct uses, as well as use cases acting through reverse dependencies.

[Etienne Dysli Metref (etienne-dysli-metref)]

Hi Robie,

Thank you for taking the time to review this SRU. I've considered the use cases of Shibboleth packages and searched for reverse dependencies and here is what I can say.

All five source packages are maintained by Shibboleth project developers as components of the Shibboleth Service Provider. Usage of the libraries for other purposes is generally not supported.

# log4shib
 - It's a library, so there are no direct use cases.
 - This new major version has a new SONAME so it is distinct from the older version.
 - Reverse dependencies: moonshot-gss-eap depends on libshibresolver1 which in turn depends on liblog4shib1v5. libshibresolver1 should keep using the current liblog4shib1v5 instead of the newer liblog4shib2. I cannot further comment on the impact upon Moonshot-related packages, but I can ask their Debian maintainer if needed.

    $ reverse-depends -r bionic src:log4shib
    Reverse-Depends
    ===============
    * libsaml9 (for liblog4shib1v5)
    * libshibresolver1 (for liblog4shib1v5)
    * libshibsp7 (for liblog4shib1v5)
    * libxmltooling-dev (for liblog4shib-dev)
    * libxmltooling7 (for liblog4shib1v5)
    * opensaml2-tools (for liblog4shib1v5)
    * shibboleth-sp2-utils (for liblog4shib1v5)

# xml-security-c
 - It's (mostly) a library, so no direct uses cases are expected. A few utility programs are shipped in xml-security-c-utils (/usr/bin/xsec-*), however these are not used for operating a Shibboleth SP. I don't have data on direct uses of these utilities.
 - This new major version has a new SONAME so it is distinct from the older version.
 - There are no reverse dependencies outside of Shibboleth packages.

    $ reverse-depends -r bionic src:xml-security-c
    Reverse-Depends
    ===============
    * libsaml9 (for libxml-security-c17v5)
    * libshibsp7 (for libxml-security-c17v5)
    * libxmltooling-dev (for libxml-security-c-dev)
    * libxmltooling7 (for libxml-security-c17v5)

# xmltooling
 - It's a library, so there are no direct use cases.
 - This new major version has a new SONAME so it is distinct from the older version.
 - Reverse dependencies: moonshot-gss-eap and libshibresolver1 both depend on libxmltooling7. The same comment as above applies for Moonshot-related packages.

    $ reverse-depends -r bionic src:xmltooling
    Reverse-Depends
    ===============
    * libapache2-mod-shib2 (for libxmltooling7)
    * libsaml2-dev (for libxmltooling-dev)
    * libsaml9 (for libxmltooling7)
    * libshibresolver1 (for libxmltooling7)
    * libshibsp-dev (for libxmltooling-dev)
    * libshibsp-plugins (for libxmltooling7)
    * libshibsp7 (for libxmltooling7)
    * libshibsp7 (for xmltooling-schemas)
    * moonshot-gss-eap (for libxmltooling7)
    * opensaml2-tools (for libxmltooling7)
    * shibboleth-sp2-utils (for libxmltooling7)

# opensaml
 - It's (mostly) a library, s...

[Etienne Dysli Metref (etienne-dysli-metref)]

> I cannot further comment on the impact upon Moonshot-related packages, but I can ask their Debian maintainer if needed.

So I asked Sam Hartman on the pkg-shibboleth-devel list and he replied:

> So, there was a new release of shibboleth-resolver along with the 3.x SP.
> I'm not sure whether the 2.x version works with the 3.x libs or not.
> There are no changes with moonshot-gss-eap for the new stack.
> It's probably desirable to rebuild the moonshot binaries, because C++ ABIs may not be entirely stable, but no source changes should be required.

Scott Cantor added that there is "not a chance" that the old shibboleth-resolver works with the SPv3 stack.

So:
 1. shibboleth-resolver 3.0.0 should also be part of this SRU, otherwise the version currently in bionic will stop working.
 2. moonshot-gss-eap needs a rebuild but would not be affected otherwise.

[Etienne Dysli Metref (etienne-dysli-metref)]

Regarding #10:
> Usage of the libraries for other purposes is generally not supported.

"not supported" here means not supported by upstream developers -- i.e. the Shibboleth project -- and isn't meant as a license to break other packages in bionic.

[Etienne Dysli Metref (etienne-dysli-metref)]

I've looked at the changes (git log) in opensaml2-tools and xml-security-c-utils to find out whether the programs they provide had changed, but apparently there are only internal changes, nothing changing the CLI. The man pages didn't change either.

Neither the release notes for xml-security-c 2.0.x (http://santuario.apache.org/creleasenotes.html) nor those for opensaml 3.0.0 (https://issues.shibboleth.net/jira/secure/ReleaseNote.jspa?projectId=10032) hint at CLI changes.

[Robie Basak (racb)]

What about Cosmic? Note that from SRU policy (https://wiki.ubuntu.com/StableReleaseUpdates), "To avoid regressions on upgrade, any such feature must then also be added to any newer supported Ubuntu release."

I'm afraid that this is going to be too time consuming for me to review - there seem to be additional complications the more I look into it (eg. Cosmic and the new soname as you mention above). Based on previous experience I think that the technical difficulties in landing this safely to existing 18.04 users are going to overwhelm the available volunteer time available from developers who are able to review it.

You might be better off maintaining a PPA for users on 18.04, or just recommending the use of Disco once it's released, combined with suitable automation, tests and CI to ensure that you can roll forward on a six monthly basis until the next LTS is released. If that seems hard to you, updating 18.04 seems harder to me.

However I welcome other Ubuntu developers to take a look if they want to help you getting this landed. I reserve judgement on this being suitable from an SRU policy perspective. If you do manage to get sponsorship, keep in mind that finding an SRU team member to review it all again is still going to a significant challenge.

I've added bug tasks for Bionic and Cosmic - getting the statuses all correct would be helpful if you want to proceed.

I'm sorry I can't help you further. I hope this doesn't discourage you from continuing to help with Shobboleth packaging in Ubuntu. Appearing as a newcomer wanting to do major surgery is your challenge here. I hope that you'll find that maintaining packaging in the development release, and landing routine bugfixes in stable releases are much easier.

[Etienne Dysli Metref (etienne-dysli-metref)]

On 15/04/2019 16.51, Robie Basak wrote:
> I'm afraid that this is going to be too time consuming for me to
> review - there seem to be additional complications the more I look
> into it (eg. Cosmic and the new soname as you mention above). Based
> on previous experience I think that the technical difficulties in
> landing this safely to existing 18.04 users are going to overwhelm
> the available volunteer time available from developers who are able
> to review it.

You are right that Cosmic must also be taken into account. However, the
situation on Cosmic is much simpler: it already has the SPv3 stack so
the upgrade would boil down to 3.0.2 -> 3.0.4. The only small issue it
that Cosmic is still using the old package names with "2" in them (i.e.
"shibboleth-sp2"), but that can be dealt with Breaks/Replaces to ensure
a smooth upgrade (as I've already done for Debian backports). Is there
something else complicating the SRU for Cosmic you were thinking of?

Can you explain how the new soname is a problem? I think it clearly
separates the new and old libraries.

> You might be better off maintaining a PPA for users on 18.04, or
> just recommending the use of Disco once it's released, combined with
> suitable automation, tests and CI to ensure that you can roll forward
> on a six monthly basis until the next LTS is released. If that seems
> hard to you, updating 18.04 seems harder to me.

Users of the Shibboleth SP software are typically web server operators,
as it is installed alongside Apache httpd. These people completely
ignore non-LTS releases and it is already hard enough to get them to
upgrade from one LTS to the next before its support expires. I've had
way more requests to backport the SPv3 stack to Xenial than I've got for
Bionic (for our PPA at http://pkg.switch.ch/switchaai/). Therefore, I
think it is unrealistic to ask server operators to upgrade their whole
OS every six months just to get a new SP version. I think it is worse to
leave Bionic with a broken Shibboleth SP for four more years than
upgrading it and risk breaking Moonshot (which can be fixed with a
no-change rebuild).

Those who have the "suitable automation, tests and CI" have already
moved past Bionic, I suppose. I want to do something for the rest out there.

> However I welcome other Ubuntu developers to take a look if they want
> to help you getting this landed.

Could you please circulate this internally so someone else may see and
tackle it?

> I've added bug tasks for Bionic and Cosmic - getting the statuses
> all correct would be helpful if you want to proceed.

What would be the correct status then?

> I'm sorry I can't help you further. I hope this doesn't discourage
> you from continuing to help with Shibboleth packaging in Ubuntu.
> Appearing as a newcomer wanting to do major surgery is your challenge
> here. I hope that you'll find that maintaining packaging in the
> development release, and landing routine bugfixes in stable releases
> are much easier.

It is indeed a daunting task for a newcomer. I submitted one security
patch for Shibboleth just before and it went well so I figured I'd
embark on a larger endeavour. :) For me, how packages are maintained in
...

[Etienne Dysli Metref (etienne-dysli-metref)]

Anyway, thank you very much Robie for your help so far! :D I really appreciate it.

[Robie Basak (racb)]

On Tue, Apr 16, 2019 at 07:23:36AM -0000, Etienne Dysli Metref wrote:
> On 15/04/2019 16.51, Robie Basak wrote:
> > I'm afraid that this is going to be too time consuming for me to
> > review - there seem to be additional complications the more I look
> > into it (eg. Cosmic and the new soname as you mention above). Based
> > on previous experience I think that the technical difficulties in
> > landing this safely to existing 18.04 users are going to overwhelm
> > the available volunteer time available from developers who are able
> > to review it.
>
> You are right that Cosmic must also be taken into account. However, the
> situation on Cosmic is much simpler: it already has the SPv3 stack so
> the upgrade would boil down to 3.0.2 -> 3.0.4. The only small issue it
> that Cosmic is still using the old package names with "2" in them (i.e.
> "shibboleth-sp2"), but that can be dealt with Breaks/Replaces to ensure
> a smooth upgrade (as I've already done for Debian backports). Is there
> something else complicating the SRU for Cosmic you were thinking of?

No, but that's still a whole lot of extra uploads, and package renaming
is another task that needs to be done right.

> Can you explain how the new soname is a problem? I think it clearly
> separates the new and old libraries.

We can't delete the old library from Bionic, so the new and old must
exist concurrently. Therefore you can't just upload a replacement source
package, because that would make a security fix or regular update for
the old library impossible[1]. We call this situation "NBS", or "not
built from any source". You'll need to create a new parallel source
package with a different name (for example with a version number in it),
so that's another whole bunch of renaming.

> > You might be better off maintaining a PPA for users on 18.04, or
> > just recommending the use of Disco once it's released, combined with
> > suitable automation, tests and CI to ensure that you can roll forward
> > on a six monthly basis until the next LTS is released. If that seems
> > hard to you, updating 18.04 seems harder to me.
>
> Users of the Shibboleth SP software are typically web server operators,
> as it is installed alongside Apache httpd. These people completely
> ignore non-LTS releases and it is already hard enough to get them to
> upgrade from one LTS to the next before its support expires. I've had
> way more requests to backport the SPv3 stack to Xenial than I've got for
> Bionic (for our PPA at http://pkg.switch.ch/switchaai/). Therefore, I
> think it is unrealistic to ask server operators to upgrade their whole
> OS every six months just to get a new SP version. I think it is worse to
> leave Bionic with a broken Shibboleth SP for four more years than
> upgrading it and risk breaking Moonshot (which can be fixed with a
> no-change rebuild).
>
> Those who have the "suitable automation, tests and CI" have already
> moved past Bionic, I suppose. I want to do something for the rest out
> there.

I understand, and I don't intend to block you from fixing Bionic now.
It's just an unrealistic amount of work, in my opinion, both to do and
to review, and I can't justify spending the time to revie...

[Etienne Dysli Metref (etienne-dysli-metref)]

On 16/04/2019 11.31, Robie Basak wrote:
>> Can you explain how the new soname is a problem? I think it
>> clearly separates the new and old libraries.
>
> We can't delete the old library from Bionic, so the new and old must
> exist concurrently. Therefore you can't just upload a replacement
> source package, because that would make a security fix or regular
> update for the old library impossible[1]. We call this situation
> "NBS", or "not built from any source". You'll need to create a new
> parallel source package with a different name (for example with a
> version number in it), so that's another whole bunch of renaming.

Ah I see now, thanks. Short of forcing every Bionic user to upgrade to
the new libraries and removing the old ones from the archive, that's an
untenable position.

> The effort really needed to have been spent while Bionic was in
> development. That would have been at least an order of magnitude
> less effort.

Yes, removing Shibboleth packages from Bionic would have been better.
Absent packages don't block backports. ;)

> I honestly think that it would be more beneficial to spend that
> effort on future development (both upstream and in distributions)
> and maintain an out-of-archive backport for Bionic users, than spend
> this disproportionate amount of effort on updating Bionic now.

Agreed. I'm already maintaining the out-of-archive backports, but I'd
like to streamline it more so it feels like using [LTS]-backports (no
special versions like ...-1switchaai1~bionic1, for example).

In the Shibboleth case, I know that upstream is contemplating packaging
for Debian and Ubuntu, but their support policy of "only the latest
version is supported" is really at odds with the current policy of both
Linux distributions.

> For example, you could add dep8 tests which would automatically block
> package updates in Ubuntu unless they pass, preventing the release of
> broken Shibboleth packages from happening again.

Thanks for the idea! I'll look into that.

> Packages are team maintained. Like Debian, use (and choice) of a VCS
> is a team decision, and the archive itself is the single source of
> truth. The default is none. As there isn't a Shibboleth team in
> Ubuntu at the moment, you are welcome to form one, create a VCS
> somewhere (though not using Launchpad or Salsa would be swimming
> against the tide for Ubuntu developers), and ask that others use it.
> In the absence of a specific team, these packages currently fall
> under the remit of the MOTU team, but I'm not aware that anyone from
> MOTU has ever worked on them, so I don't think there would be any
> conflict there.

IMHO, ideally, the Shibboleth packaging repositories over on salsa.d.o
[1] should be the reference also for Ubuntu packaging. I can create an
ubuntu/ branch namespace there for this purpose, as per DEP-14. If I
understand you correctly, creating a team on LP would be enough to
"claim" ownership of these packages, though not enough to force others
to use salsa before uploading to Ubuntu.

I'll go ahead and create a team on LP as that seems to be a step in the
right direction. Then I'll try to reflect the current state of Ubuntu
packages in the Git rep...

[Robie Basak (racb)]

[moving this thread over to ubuntu-motu@ from
https://launchpad.net/bugs/1822069]

On Tue, Apr 16, 2019 at 11:28:58AM -0000, Etienne Dysli Metref wrote:
> IMHO, ideally, the Shibboleth packaging repositories over on salsa.d.o
> [1] should be the reference also for Ubuntu packaging. I can create an
> ubuntu/ branch namespace there for this purpose, as per DEP-14.

That sounds great, and fits a common pattern used by other Ubuntu
development teams too.

When an Ubuntu delta exists, you can adjust Vcs-* in debian/control to
point to the correct place on Salsa to help other Ubuntu developers find
the correct VCS.

Note that Ubuntu developers may upload (to Ubuntu) without updating the
VCS first. This is rather like an NMU in Debian, but more common. For
example, in Ubuntu uploads needed for library transitions are generally
driven from the library provider end, rather than the library consumer
end, and so uploads relating to a transition may just "appear" in the
archive. If this happens, you'll need to pull in those uploads to your
VCS, as the archive remains the single source of truth.

However, if you're available and developers know to talk to you, they'll
try to avoid stepping on your toes as much as possible by communicating
first.

> If I
> understand you correctly, creating a team on LP would be enough to
> "claim" ownership of these packages, though not enough to force others
> to use salsa before uploading to Ubuntu.

Right. There's no ownership claim as such; just an understanding that
your team is volunteering to generally look after the packages, and
others will try to work with you on that.

> I'll go ahead and create a team on LP as that seems to be a step in the
> right direction. Then I'll try to reflect the current state of Ubuntu
> packages in the Git repositories on salsa under ubuntu/ (changelog,
> patches, etc.) so that all of this can be unified. I'd still need
> sponsorship to upload though, but that can be sorted later.

I suggest you announce your intentions to ubuntu-devel@ and
ubuntu-motu@, together with details on how to contact your team (perhaps
nominate ubuntu-motu@ for now, and subscribe).

In fact, we're quite far off topic on this bug now. I suggest we switch
to talking on ubuntu-motu@ instead. I'll Cc: that list now, and switch
the bug to Bcc:, to move this conversation over there.

[Etienne Dysli Metref (etienne-dysli-metref)]

version in disco is the target one

[Etienne Dysli Metref (etienne-dysli-metref)]

source package was renamed opensaml2 -> opensaml

[Etienne Dysli Metref (etienne-dysli-metref)]

source package was renamed shibboleth-sp2 -> shibboleth-sp

[Etienne Dysli Metref (etienne-dysli-metref)]

version in disco is the target one

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.