[MIR] zeromq3

[Sebastien Bacher]

Trying to get unity8 in main this cycle, that's one of the depends of unity-scopes-api (MIR to come)

* availability
it's available/built on all the ubuntu architectures, https://launchpad.net/ubuntu/+source/zeromq3/4.1.4-7

* security
the trusty version has some open CVEs which seem to have been resolved in the newer versions/series of Ubuntu
http://people.canonical.com/~ubuntu-security/cve/pkg/zeromq3.html

* quality
- the package is well maintained in Debian
- it works out of the box with no configuration required
- it has a testsuite which is used during build but currently has some errors and doesn't stop the build

* dependencies
requires libsodium which is universe

* standards compliance
FHS and Debian Policy compliant.

* ubuntu maintainance
the desktop team is going to look after it, desktop-bugs has been subscribed

* background
no specific info

[Łukasz Zemczak (sil2100)]

Hey! Just wanted to give a heads up - we are trying to get an FFe for a snapshot-release of zeromq3 (pre-4.2) to yakkety for the scopes-api needs. The bug for tracking this is here:

https://bugs.launchpad.net/ubuntu/+source/zeromq3/+bug/1612240

[Matthias Klose (doko)]

maybe unrelated, but the python bindings (pyzmq) currently fail to build on s390x

[Łukasz Zemczak (sil2100)]

Will be looking into the pyzmq failures once we sync up the new zeromq3, as I want to do some work there related to the newly exported constants.

[Seth Arnold (seth-arnold)]

I reviewed zeromq3 version 4.1.5-2 as checked into yakkety. This shouldn't
be considered a full audit but rather a quick check of maintainability.

zmq is a networking and related utilities library. However, it has a
broad, almost obsessive, vision of how the sockets API should look;
calling it a "networking library" is entirely underselling the intention.

- Build-Depends: debhelper, dh-autoreconf, libpgm-dev, libsodium-dev,
  pkg-config
- Does not itself daemonize
- Does not itself direct networking
- No pre/post inst/rm scripts
- No initscripts
- No dbus services
- No setuid executables
- No binaries in the path
- No sudo fragments
- No udev rules
- Tests run during build -- six failures in Xenial build logs and seven
  failures in yakkety build logs -- why do these fail? why do they not
  halt the build?
- No cronjobs
- Build logs look clean beyond the test failures

- No spawned subprocesses
- Memory management is old-school correct-to-the-byte-style C-flavor. It
  all looks careful but is very manual, with hand-counted bytes and
  detailed knowledge of packet layouts necessary to make any
  modifications. As this is a defacto protocol, changes shouldn't be
  necessary, but there is no margin of safety.
- Only /dev/urandom file IO, looked very careful
- Logging looked safe
- Slight use of ioctls
- Uses libsodium or tweetnacl for newer cryptographically secured
  mechanisms
- Extensive networking -- this is easily the most complicated networking
  code I've ever reviewed. It all looks careful but expert assistance
  would be needed for nearly any modifications. (The theory of the library
  is that it would handle all the complicated portions of networking and
  allow applications to focus on protocol design. The complexity is
  expected and appropriate.)
- No portions of code are more privileged than others
- No temporary file handling
- No webkit
- No policykit
- No javascript
- Clean cppcheck

zmq is extremely ambitious; it embraces complexity so that client
programs can be written more simply. The enthusiasm and reach and
optimism are infectious. The code quality is extremely high, even
though it does work with razor-thin margins for memory management, and
must manage the full complexities of many networking protocols on many
operating systems. Error-checking is pervasive and careful. Comments
are clear and meaningful. It's a striking and bold new mechanism to
network everything, simply.

This comes at a cost.

I found two bugs in our package that have been fixed upstream. (Bugs
1622073 and 1623792.) I didn't try to write programs to discover the
consequences of tripping these bugs but program death is expected. We
should fix these.

Yakkety zeromq3 packages have seven failures in the test suite run
during the build. The build is not aborted with these failures. We need
to understand, and if feasible, fix these failed tests. Ideally the build
would fail when the tests fail, so that failures do not become normal.

Security team provides conditional approval for promoting zeromq3 to
main -- please fix the test failures or explain why they cannot be fixed
before promoting the package.

Thanks

[Michael Terry (mterry)]

From a packaging/maintaining side, I agree that zeromq3 looks fine besides the tests.

In particular, we should absolutely not run the test suite with the '-' in front of it in debian/rules which ignores the return value. In the worst case, failing tests should be skipped (but that's still bad!). But we definitely shouldn't throw out the good tests with the bad ones, if we can help it.

I'm separately also curious about fixing the tests that are failing currently.

[Sebastien Bacher (seb128)]

Łukasz was working on an update so assigning to him to look at those issues

[Steve Langasek (vorlon)]

zeromq3 in yakkety-proposed now enforces the test suite, and passes with fixes to make the test suite not use hard-coded local ports.

   https://launchpad.net/ubuntu/+source/zeromq3/4.1.5-2ubuntu3

(Bad idea to hard-code local port numbers in a test suite. Really bad idea to do so with the same local port numbers, when your tests are run in parallel.)

Łukasz, I leave it to you to forward-port this to the current upstream git and forward to upstream.

[Sebastien Bacher (seb128)]

the bug should be good for the MIR team to have another look then

[Michael Terry (mterry)]

Thank you Steve!

[Łukasz Zemczak (sil2100)]

Please note that I will be publishing a new git-snapshot based version of zeromq3 today as we have been granted the FFe: https://bugs.launchpad.net/ubuntu/+source/zeromq3/+bug/1612240

All the concerns raised here have been addressed there as well, with all Steve's changes forwarded to the new packages.

[Steve Langasek (vorlon)]

Override component to main
zeromq3 4.1.5-2 in yakkety: universe/libs -> main
libzmq3-dev 4.1.5-2 in yakkety amd64: universe/libdevel/optional/100% -> main
libzmq3-dev 4.1.5-2 in yakkety arm64: universe/libdevel/optional/100% -> main
libzmq3-dev 4.1.5-2 in yakkety armhf: universe/libdevel/optional/100% -> main
libzmq3-dev 4.1.5-2 in yakkety i386: universe/libdevel/optional/100% -> main
libzmq3-dev 4.1.5-2 in yakkety powerpc: universe/libdevel/optional/100% -> main
libzmq3-dev 4.1.5-2 in yakkety ppc64el: universe/libdevel/optional/100% -> main
libzmq3-dev 4.1.5-2 in yakkety s390x: universe/libdevel/optional/100% -> main
libzmq5 4.1.5-2 in yakkety amd64: universe/libs/optional/100% -> main
libzmq5 4.1.5-2 in yakkety arm64: universe/libs/optional/100% -> main
libzmq5 4.1.5-2 in yakkety armhf: universe/libs/optional/100% -> main
libzmq5 4.1.5-2 in yakkety i386: universe/libs/optional/100% -> main
libzmq5 4.1.5-2 in yakkety powerpc: universe/libs/optional/100% -> main
libzmq5 4.1.5-2 in yakkety ppc64el: universe/libs/optional/100% -> main
libzmq5 4.1.5-2 in yakkety s390x: universe/libs/optional/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety amd64: universe/debug/extra/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety arm64: universe/debug/extra/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety armhf: universe/debug/extra/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety i386: universe/debug/extra/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety powerpc: universe/debug/extra/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety ppc64el: universe/debug/extra/100% -> main
libzmq5-dbg 4.1.5-2 in yakkety s390x: universe/debug/extra/100% -> main
22 publications overridden.

[Christian Ehrhardt  (paelzer)]

I filed bug 1820232 (now a dup) only to realize this was already approved.
Under the usual terms I'd want the ack on this MIR to be used to (re-)promote it once we pull it back in by seeding mailman3.