[SRU] ceph 10.2.2

[James Page]

>>> Test Plan

taken from OpenStack team wiki page

"Once stable package updates have been accepted by the ubuntu-sru (or Cloud Archive) team into -proposed pockets, the following SRU verification process is followed:

Deployment and base configuration using OpenStack Charm Testing bundles and charms, using the current set of stable charms configured to consume packages from the proposed pocket of the archive.

Testing of the deployed Cloud using the Tempest (the OpenStack functional test project) smoke test target; this is approximately 100 tests from the full Tempest upstream function test suite that cover all core functions of the cloud. The deployed cloud is expected to pass all smoke tests.

For updates where there is risk of regression as a result of the package upgrade process, the same testing process is followed as above, deploying from archive excluding proposed, testing using Tempest, upgrading the deployed cloud to proposed and then re-verifying the cloud using Tempest.

This testing process is automated by the Ubuntu OpenStack CI system.

Additionally, any specific test cases covered in SRU bug reports should be explicitly tested as well.
"

>>>>
This is the first bugfix release for Jewel. It contains several annoying packaging and init system fixes and a range of important bugfixes across RBD, RGW, and CephFS.

We recommend that all v10.2.x users upgrade.

For more detailed information, see the complete changelog.

NOTABLE CHANGES

cephfs: CephFSVolumeClient should isolate volumes by RADOS namespace (issue#15400, pr#8787, Xiaoxi Chen)
cephfs: handle standby-replay nodes properly in upgrades (issue#15591, pr#8971, John Spray)
ceph-{mds,mon,osd} packages need scriptlets with systemd code (issue#14941, pr#8801, Boris Ranto, Nathan Cutler)
ceph_test_keyvaluedb: fix (issue#15435, pr#9051, Allen Samuels, Sage Weil)
cmake: add missing source file to rbd_mirror/image_replayer (pr#9052, Casey Bodley)
cmake: fix rbd compile errors (pr#9076, runsisi, Jason Dillaman)
journal: incorrectly computed object offset within set (issue#15765, pr#9038, Jason Dillaman)
librbd: client-side handling for incompatible object map sizes (issue#15642, pr#9039, Jason Dillaman)
librbd: constrain size of AioWriteEvent journal entries (issue#15750, pr#9048, Jason Dillaman)
librbd: does not crash if image header is too short (pr#9044, Kefu Chai)
librbd: Errors encountered disabling object-map while flatten is in-progress (issue#15572, pr#8869, Jason Dillaman)
librbd: fix get/list mirror image status API (issue#15771, pr#9036, Mykola Golub)
librbd: Parent image is closed twice if error encountered while opening (issue#15574, pr#8867, Jason Dillaman)
librbd: possible double-free of object map invalidation request upon error (issue#15643, pr#8865, runsisi)
librbd: possible race condition leads to use-after-free (issue#15690, pr#9009, Jason Dillaman)
librbd: potential concurrent event processing during journal replay (issue#15755, pr#9040, Jason Dillaman)
librbd: Potential double free of SetSnapRequest instance (issue#15571, pr#8803, runsisi)
librbd: put the validation of image snap context earlier (pr#9046, runsisi)
librbd: reduce log level for image format 1 warning (issue#15577, pr#9003, Jason Dillaman)
mds/MDSAuthCap parse no longer fails on paths with hyphens (issue#15465, pr#8969, John Spray)
mds: MDS incarnation no longer gets lost after remove filesystem (issue#15399, pr#8970, John Spray)
mon/OSDMonitor: avoid underflow in reweight-by-utilization if max_change=1 (issue#15655, pr#9006, Samuel Just)
python: clone operation will fail if config overridden with “rbd default format = 1” (issue#15685, pr#8972, Jason Dillaman)
radosgw-admin: add missing –zonegroup-id to usage (issue#15650, pr#9019, Casey Bodley)
radosgw-admin: update usage for zone[group] modify (issue#15651, pr#9016, Casey Bodley)
radosgw-admin: zonegroup remove command (issue#15684, pr#9015, Casey Bodley)
rbd CLI to retrieve rbd mirror state for a pool / specific image (issue#15144, issue#14420, pr#8868, Mykola Golub)
rbd disk-usage CLI command should support calculating full image usage (issue#14540, pr#8870, Jason Dillaman)
rbd: helpful error message on map failure (issue#15721, pr#9041, Venky Shankar)
rbd: help message distinction between commands and aliases (issue#15521, pr#9004, Yongqiang He)
rbd-mirror: admin socket commands to start/stop/restart mirroring (issue#15718, pr#9010, Mykola Golub, Josh Durgin)
rbd-mirror can crash if start up is interrupted (issue#15630, pr#8866, Jason Dillaman)
rbd-mirror: image sync needs to handle snapshot size and protection status (issue#15110, pr#9050, Jason Dillaman)
rbd-mirror: lockdep error during bootstrap (issue#15664, pr#9008, Jason Dillaman)
rbd-nbd: fix rbd-nbd aio callback error handling (issue#15604, pr#9005, Chang-Yi Lee)
rgw: add AWS4 completion support for RGW_OP_SET_BUCKET_WEBSITE (issue#15626, pr#9018, Javier M. Mellid)
rgw admin output (issue#15747, pr#9054, Casey Bodley)
rgw: fix issue #15597 (issue#15597, pr#9020, Yehuda Sadeh)
rgw: fix printing wrong X-Storage-Url in Swift’s TempAuth. (issue#15667, pr#9021, Radoslaw Zarzynski)
rgw: handle stripe transition when flushing final pending_data_bl (issue#15745, pr#9053, Yehuda Sadeh)
rgw: leak fixes (issue#15792, pr#9022, Yehuda Sadeh)
rgw: multisite: Issues with Deleting Buckets (issue#15540, pr#8930, Abhishek Lekshmanan)
rgw: period commit fix (issue#15828, pr#9081, Casey Bodley)
rgw: period delete fixes (issue#15469, pr#9047, Casey Bodley)
rgw: radosgw-admin zone set cuts pool names short if name starts with a period (issue#15598, pr#9029, Yehuda Sadeh)
rgw: segfault at RGWAsyncGetSystemObj (issue#15565, issue#15625, pr#9017, Yehuda Sadeh)
several backports (issue#15588, issue#15655, pr#8853, Alexandre Derumier, xie xingguo, Alfredo Deza)
systemd: fix typo in preset file (pr#8843, Nathan Cutler)
tests: make check fails on ext4 (issue#15837, pr#9063, Loic Dachary, Sage Weil)

This point release fixes several important bugs in RBD mirroring, RGW multi-site, CephFS, and RADOS.

We recommend that all v10.2.x users upgrade.

For more detailed information, see the complete changelog.

NOTABLE CHANGES

ceph: cli: exception when pool name has non-ascii characters (issue#15913, pr#9320, Ricardo Dias)
ceph-disk: workaround gperftool hang (issue#13522, issue#16103, pr#9427, Loic Dachary)
cephfs: backports needed for Manila (issue#15599, issue#15417, issue#15045, pr#9430, John Spray, Ramana Raja, Xiaoxi Chen)
ceph.spec.in: drop support for RHEL<7 and SUSE<1210 in jewel and above (issue#15725, issue#15627, issue#13445, issue#15822, issue#15472, issue#15987, issue#15516, issue#15549, pr#8938, Boris Ranto, Sage Weil, Nathan Cutler, Lars Marowsky-Bree)
ceph_test_librbd_fsx crashes during journal replay shut down (issue#16123, pr#9556, Jason Dillaman)
client: fix bugs accidentally disabling readahead (issue#16024, pr#9656, Patrick Donnelly, Greg Farnum)
cls_journal: initialize empty commit position upon client register (issue#15757, pr#9376, runsisi, Venky Shankar)
cls::rbd: mirror_image_status_list returned max 64 items (pr#9069, Mykola Golub)
cls_rbd: mirror image status summary should read full directory (issue#16178, pr#9608, Jason Dillaman)
common: BackoffThrottle spins unnecessarily with very small backoff while the throttle is full (issue#15953, pr#9579, Samuel Just)
common: Do not link lttng into libglobal (pr#9194, Karol Mroz)
debian: install systemd target files (issue#15573, pr#8815, Kefu Chai, Sage Weil)
doc: update mirroring guide to include pool/image status commands (issue#15746, pr#9180, Mykola Golub)
librbd: Disabling journaling feature results in “Transport endpoint is not connected” error (issue#15863, pr#9548, Yuan Zhou)
librbd: do not shut down exclusive lock while acquiring’ (issue#16291, issue#16260, pr#9691, Jason Dillaman)
librbd: Initial python APIs to support mirroring (issue#15656, pr#9550, Mykola Golub)
librbd: journal IO error results in failed assertion in AioCompletion (issue#16077, issue#15034, issue#15791, pr#9611, Hector Martin, Jason Dillaman)
librbd: journal: live replay might skip entries from previous object set (issue#15864, issue#15665, pr#9217, Jason Dillaman)
librbd: journal: support asynchronous shutdown (issue#15949, issue#14530, issue#15993, pr#9373, Jason Dillaman)
librbd: Metadata config overrides are applied synchronously (issue#15928, pr#9318, Jason Dillaman)
librbd: Object Map is showing as invalid, even when Object Map is disabled for that Image. (issue#16076, pr#9555, xinxin shu)
librbd: prevent error messages when journal externally disabled (issue#16114, pr#9610, Zhiqiang Wang, Jason Dillaman)
librbd: recursive lock possible when disabling journaling (issue#16235, pr#9654, Jason Dillaman)
librbd: refresh image if needed in mirror functions (issue#16096, pr#9609, Jon Bernard)
librbd: remove should ignore mirror errors from older OSDs (issue#16268, pr#9692, Jason Dillaman)
librbd: reuse ImageCtx::finisher and SafeTimer for lots of images case (issue#13938, pr#9580, Haomai Wang)
librbd: validate image metadata configuration overrides (issue#15522, pr#9554, zhuangzeqiang)
mds: order directories by hash and fix simultaneous readdir races (issue#15508, pr#9655, Yan, Zheng, Greg Farnum)
mon: Hammer (0.94.3) OSD does not delete old OSD Maps in a timely fashion (maybe at all?) (issue#13990, pr#9100, Kefu Chai)
mon/Monitor: memory leak on Monitor::handle_ping() (issue#15793, pr#9270, xie xingguo)
osd: acting_primary not updated on split (issue#15523, pr#8968, Sage Weil)
osd: boot race with noup being set (issue#15678, pr#9101, Sage Weil)
osd: deadlock in OSD::_committed_osd_maps (issue#15701, pr#9103, Xinze Chi)
osd: hobject_t::get_max() vs is_max() discrepancy (issue#16113, pr#9614, Samuel Just)
osd: LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 segv (issue#15760, pr#9104, Sage Weil)
osd: remove reliance on FLAG_OMAP for reads (pr#9638, Samuel Just)
osd valgrind invalid reads/writes (issue#15870, pr#9237, Samuel Just)
pybind: rbd API should default features parameter to None (issue#15982, pr#9553, Mykola Golub)
qa: dynamic_features.sh races with image deletion (issue#15500, pr#9552, Mykola Golub)
qa/workunits: ensure replay has started before checking position (issue#16248, pr#9674, Jason Dillaman)
qa/workunits/rbd: fixed rbd_mirror teuthology runtime errors (pr#9232, Jason Dillaman)
radosgw-admin: fix ‘period push’ handling of –url (issue#15926, pr#9210, Casey Bodley)
rbd-mirror: Delete local image mirror when remote image mirroring is disabled (issue#15916, issue#14421, pr#9372, runsisi, Mykola Golub, Ricardo Dias)
rbd-mirror: do not propagate deletions when pool unavailable (issue#16229, pr#9630, Jason Dillaman)
rbd-mirror: do not re-use image id from mirror directory if creating image (issue#16253, pr#9673, Jason Dillaman)
rbd-mirror: FAILED assert(!m_status_watcher) (issue#16245, issue#16290, pr#9690, Mykola Golub)
rbd-mirror: fix deletion propagation edge cases (issue#16226, pr#9629, Jason Dillaman)
rbd-mirror: fix journal shut down ordering (issue#16165, pr#9628, Jason Dillaman)
rbd-mirror: potential crash during image status update (issue#15909, pr#9226, Mykola Golub, Jason Dillaman)
rbd-mirror: refresh image after creating sync point (issue#16196, pr#9627, Jason Dillaman)
rbd-mirror: replicate cloned images (issue#14937, pr#9423, Jason Dillaman)
rbd-mirror should disable the rbd cache for local images (issue#15930, pr#9317, Jason Dillaman)
rbd-mirror: support bootstrap canceling (issue#16201, pr#9612, Mykola Golub)
rbd-mirror: support multiple replicated pools (issue#16045, pr#9409, Jason Dillaman)
rgw: fix manager selection when APIs customized (issue#15974, issue#15973, pr#9245, Robin H. Johnson)
rgw: keep track of written_objs correctly (issue#15886, pr#9239, Yehuda Sadeh)
rpm: ceph gid mismatch on upgrade from hammer with pre-existing ceph user (SUSE) (issue#15869, pr#9424, Nathan Cutler)
systemd: ceph-{mds,mon,osd,radosgw} systemd unit files need wants=time-sync.target (issue#15419, pr#8802, Nathan Cutler)
test: failure in journal.sh workunit test (issue#16011, pr#9377, Mykola Golub)
tests: rm -fr /tmp/virtualenv (issue#16087, pr#9403, Loic Dachary)

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceph - 10.2.1-0ubuntu1

---------------
ceph (10.2.1-0ubuntu1) yakkety; urgency=medium

  * New upstream version (LP: #1585660).
    - d/p/drop-user-group-osd-prestart.patch: Dropped, included upstream.
  * d/rules,ceph-common.install: Ensure that /etc/default/ceph is a file
    and not a directory (LP: #1587516).

 -- James Page <email address hidden> Mon, 06 Jun 2016 09:23:35 +0100

[James Page (james-page)]

10.2.2 appeared before I managed to work the fix for bug 1587516; I'll re-test and re-upload for Xenial rather than work the SRU process twice for no gain.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceph - 10.2.2-0ubuntu1

---------------
ceph (10.2.2-0ubuntu1) yakkety; urgency=medium

  * New upstream version (LP: #1585660):
    - d/ceph.install: Drop install of 60-ceph-partuuid-workaround.rules
      as no longer part of upstream codebase.
  * d/control: Bumped Standards-Version to 3.9.8, no changes.

 -- James Page <email address hidden> Thu, 16 Jun 2016 17:40:04 +0100

[Martin Pitt (pitti)]

There are lot of changes here, and it's not realistic for the SRU team to vet the diff. What is the test plan for this update? Please add an SRU test case as usual (particularly around the moved conffile) and how you ensure that this won't regress existing installs. Thanks!

[Martin Pitt (pitti)]

Oh, the conffile is already bug 1587516 which has a test case. So just the normal regression test plan please.

[James Page (james-page)]

I've documented the testing process that we undertake as part of SRU verification here:

  https://wiki.ubuntu.com/OpenStack/StableReleaseUpdates

This includes the Ceph packages, which are currently managed by the Ubuntu OpenStack team.

[Martin Pitt (pitti)]

Hello James, or anyone else affected,

Accepted ceph into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/10.2.2-0ubuntu0.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris J Arges (arges)]

Hello James, or anyone else affected,

Accepted ceph into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/10.2.2-0ubuntu0.16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Martin Pitt (pitti)]

The verification of the Stable Release Update for ceph has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceph - 10.2.2-0ubuntu0.16.04.2

---------------
ceph (10.2.2-0ubuntu0.16.04.2) xenial; urgency=medium

  * d/ceph-common.{preinst,postinst,postrm}: Fix version checks in maintainer
    scripts to ensure that /etc/default/ceph actually gets installed on
    upgrade (LP: #1587516).

ceph (10.2.2-0ubuntu0.16.04.1) xenial; urgency=medium

  * New upstream stable release (LP: #1585660).
    - d/p/drop-user-group-osd-prestart.patch: Dropped, included upstream.
    - d/ceph.install: Drop install of 60-ceph-partuuid-workaround.rules, no
      longer needed and not shipped by upstream.
  * Rename /etc/default/ceph/ceph -> /etc/default/ceph (LP: #1587516):
    - d/rules,ceph-common.install: Ensure that /etc/default/ceph is a file
      and not a directory.
    - d/ceph-common.{preinst,postinst,postrm}: Ensure that rename of
      /etc/default/ceph/ceph -> /etc/default/ceph is handled correctly
      and that any end-user changes are preserved.

 -- James Page <email address hidden> Tue, 19 Jul 2016 10:32:34 +0100

[James Page (james-page)]

This bug was fixed in the package ceph - 10.2.2-0ubuntu0.16.04.2~cloud0
---------------

 ceph (10.2.2-0ubuntu0.16.04.2~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 ceph (10.2.2-0ubuntu0.16.04.2) xenial; urgency=medium
 .
   * d/ceph-common.{preinst,postinst,postrm}: Fix version checks in maintainer
     scripts to ensure that /etc/default/ceph actually gets installed on
     upgrade (LP: #1587516).