[FFe] Please upgrade zeromq3 and zmqpp to 4.1.2 git snapshot

Bug #1612240 reported by Paweł Stołowski on 2016-08-11
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
zeromq3 (Ubuntu)
High
Łukasz Zemczak
zmqpp (Ubuntu)
High
Łukasz Zemczak

Bug Description

Please update zmqpp to version 4.1.2 with the following patch that I proposed upstream (it got merged into their "develop" branch):
https://github.com/zeromq/zmqpp/pull/167/files

(alternatively, use master + my patch or develop branch of zmqpp).

This will require the current git snapshot of the zmq library (which will be version 4.2 when it's released) - https://github.com/zeromq/libzmq

All the above is needed to support the new USE_FD socket option, which will probably be critical for scopes as snaps.

== Feature Freeze Exception details ==

These packages are required by the ongoing work of the unity scopes for snappy. Due to unrelated to zeromq issues (unity-scopes-api unit-test issues on arm64, possibly due to recent toolchain changes) we did not make it in time before Feature Freeze. The packages have been prepared and tested in:

https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-005/+packages?field.name_filter=&field.status_filter=published&field.series_filter=yakkety

There is no ABI breakage in zeromq3 so no reverse-depends rebuilds are required. The zmqpp package does break ABI so a so bump and rebuild of rev-deps was needed, but the currently only consumer of the zmqpp packages is unity-scopes-api. The new release of zeromq3 is based on a git snapshot as a new stable release of this project is planned in the nearest future, so a bit too late for the current plans. Some reverse-depends of zeromq3 have been tested to work fine with the new package as is.

We kindly request a FFe for this very particular case.

Changed in zmqpp (Ubuntu):
assignee: nobody → Łukasz Zemczak (sil2100)
summary: - Please upgrade zmqpp to 4.1.2 git snapshot
+ Please upgrade zeromq3 and zmqpp to 4.1.2 git snapshot
Changed in zeromq3 (Ubuntu):
assignee: nobody → Łukasz Zemczak (sil2100)
Changed in zmqpp (Ubuntu):
status: New → Triaged
Changed in zeromq3 (Ubuntu):
status: New → In Progress
Changed in zmqpp (Ubuntu):
status: Triaged → In Progress
summary: - Please upgrade zeromq3 and zmqpp to 4.1.2 git snapshot
+ [FFe] Please upgrade zeromq3 and zmqpp to 4.1.2 git snapshot
description: updated
Changed in zeromq3 (Ubuntu):
importance: Undecided → High
Changed in zmqpp (Ubuntu):
importance: Undecided → High
description: updated
description: updated
Steve Langasek (vorlon) wrote :

Hi Łukasz,

When you say the next stable release of zeromq3 is "planned in the nearest future", when exactly is that?

Since there is also a plan to get unity8 into main properly for 16.10, this would seem to imply pulling zeromq3 into main. So we would want to make sure we can provide security support. Would we want to include the final release of zeromq3 4.2 in 16.10 under this FFe?

Changed in zeromq3 (Ubuntu):
status: In Progress → Incomplete
Łukasz Zemczak (sil2100) wrote :

Hey Steve!

So the zeromq crew was planning on starting the 4.2 release in May already, but things seem to have been pushed back and back with still no concrete release date. They're working on it still, also involving people from Debian, but there is simply no release date as of yet. But the release is pending - it's been mentioned multiple times that they're 'ready' for 4.2. My expectation is that it should be out (at least some official RC) in the nearest 1-2 months.
I will try to reach out to the developers personally to maybe check on any possible ETA's from them.

This was also one of the reasons why I have been asked to prepare a snapshot based release for now. Since the release date is a big '?' for now, the scopes-api snap work would be blocked. From the assertions that have been made by Paweł from the scopes team it was found that backporting all the required changes to 4.1.5 would be risky and could potentially cause more harm. It's a tricky situation right now.

We think of the snapshot as a temporary solution until 4.2 is released, as we'd like to sync up to an official upstream version as soon as possible. In that case we would like to get 4.2 as part of the FFe to 16.10 still.

Steve Langasek (vorlon) wrote :

Ok. If 4.2 has been in limbo since May, it's possible it won't be ready in time for 16.10 release. It's also possible that it will be released, but it will unexpectedly introduce incompatibilities with the current git snapshot. If either of these happens, what is the plan for handling this?

Łukasz Zemczak (sil2100) wrote :

There are a few possibilities. One is that we try to force an update to 4.2 no matter when it's released, trying to push it into yakkety even after release-day. The rationale would be that the snapshot is anyway closer to 4.2 than anything, so it would be more like a 'bugfix' than a new feature-release. All features planned for 4.2 are already in so there should be no surprises here.
We can of course also track the development of 4.2 and cherry-pick important fixes to our snapshot release, but I would really like to switch to 4.2 as soon as possible.

4.2 introduces no ABI breakages (at least not any obvious ones looking at the symbol files), so any step here should be relatively convenient as no rebuilds are needed (instead of zmqpp of course). It's backward compatible API-wise.

Another possibility would be, of course, simply pushing a fork of zeromq3 to the archives that would be snapshot-based. This would mean we'd stay on 4.1.5 on yakkety for zeromq3 but some separate package, like, zeromq3-future (wrong name, but just an example) would provide the snapshot-based binaries instead. This way we would only have zmqpp as the consumer (and the only consumer of zmqpp is unity-scopes-api). This is much safer but very ugly from the archive perspective - but maybe this could be thought of as a better solution?

It's also tough for me as this is the first situation like this we have to deal with.

Łukasz Zemczak (sil2100) wrote :

I now re-checked zeromq trunk to see if any critical issues have been found and fixed, but almost all the new commits in git are either Windows-related, build-specific or documentation updates. So far it looks that code-wise things are rather-stable.

Could we get a final decision here? The ubuntu-mir team said they are not concerned by the snapshotting as long as the packages pass security review.

Changed in zeromq3 (Ubuntu):
status: Incomplete → Confirmed
Steve Langasek (vorlon) wrote :

We don't want to pull this in as a completely separate source package; that doesn't address the support problem, as it's precisely the "forked" version that we would need to support.

Since there is no library transition here, and it sounds like the risk of an ABI break before 4.2 releases is quite low, and you have committed to dealing with any collateral damage, FFe granted.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package zmqpp - 4.1.2-0ubuntu1

---------------
zmqpp (4.1.2-0ubuntu1) yakkety; urgency=medium

  * New upstream release
  * debian/copyright:
    - Follow up on the license change to MPLv2
  * debian/watch:
    - Point to the new upstream location
  * debian/patches/support_use_fd_option.diff:
    - Support for ZMQ_USE_FD (LP: #1612240)
  * debian/patches/fix_ftbfs_new_boost.diff:
    - Fix FTBFS on yakkety due to boost1.61
  * debian/control:
    - Bump the libzmq3-dev dependency for the new required features in zeromq
    - Add the libsodium-dev build-dep for the test libs
    - Rename package to libzmqpp4 as the ABI changed and the soname was bumped
    - Bump the standards version to 3.9.7
  * debian/rules:
    - Remove the unneeded statically linked library archive
  * Drop debian/patches/mkdir_before_install.diff
  * Drop debian/patches/fix_1256886.diff

 -- Łukasz 'sil2100' Zemczak <email address hidden> Thu, 11 Aug 2016 17:15:27 +0200

Changed in zmqpp (Ubuntu):
status: In Progress → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package zeromq3 - 4.1.5+git20160811+2fc86bc-0ubuntu2

---------------
zeromq3 (4.1.5+git20160811+2fc86bc-0ubuntu2) yakkety; urgency=medium

  [ Łukasz 'sil2100' Zemczak ]
  * New upstream snapshot based release (2fc86bc) (LP: #1612240)
  * debian/libzmq5.symbols:
    - Update to include the new exported symbols
  * debian/patches/*:
    - Refresh patches, removed no longer needed sys_ucred_h.patch and
      hurd-support_4.1.5.patch
  * debian/copyright:
    - Updated copyright contents
  * debian/patches/no-hardcoded-port-in-tests.patch:
    - Refresh to the snapshot release of zeromq

  [ Steve Langasek ]
  * Enforce the test suite instead of ignoring test failures, as requested
    by the MIR Team (#1597439)
  * debian/patches/no-hardcoded-port-in-tests.patch: never use hard-
    coded local port numbers in tests.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Wed, 21 Sep 2016 12:32:23 +0200

Changed in zeromq3 (Ubuntu):
status: Confirmed → Fix Released
Luca Boccassi (bluca) wrote :

Hello,

Just found this discussion. We just released 4.2.0, sorry it took a while, difficult times.
I hope the delay didn't cause you guys too much trouble.

The patch to use dynamic ports in the test looks very interesting, it's something I wanted to fix for a while but never had to the time, so if you want to push it up to Github we'd have a look, although it has to work on Windows too (I see there's some pthread use). Also at least one should use the hard-coded port, to test that code path, but it can be a separate individual unit test.

Thanks for your work!

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers