[SRU] 0.80.9 point release

Bug #1413917 reported by James Page on 2015-01-23
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ceph (Ubuntu)
High
Unassigned
Trusty
High
James Page
Utopic
High
James Page
Vivid
High
Unassigned

Bug Description

This is a long-awaited bugfix release for firefly. It has several
imporant (but relatively rare) OSD peering fixes, performance issues when
snapshots are trimmed, several RGW fixes, a paxos corner case fix, and
some packaging updates.

We recommend that all users for v0.80.x firefly upgrade when it is
convenient to do so.

Notable Changes
---------------

* build: remove stack-execute bit from assembled code sections (#10114 Dan
  Mick)
* ceph-disk: fix dmcrypt key permissions (#9785 Loic Dachary)
* ceph-disk: fix keyring location (#9653 Loic Dachary)
* ceph-disk: make partition checks more robust (#9721 #9665 Loic Dachary)
* ceph: cleanly shut down librados context on shutdown (#8797 Dan Mick)
* common: add $cctid config metavariable (#6228 Adam Crume)
* crush: align rule and ruleset ids (#9675 Xiaoxi Chen)
* crush: fix negative weight bug during create_or_move_item (#9998 Pawel
  Sadowski)
* crush: fix potential buffer overflow in erasure rules (#9492 Johnu
  George)
* debian: fix python-ceph -> ceph file movement (Sage Weil)
* libcephfs,ceph-fuse: fix flush tid wraparound bug (#9869 Greg Farnum,
  Yan, Zheng)
* libcephfs: close fd befure umount (#10415 Yan, Zheng)
* librados: fix crash from C API when read timeout is enabled (#9582 Sage
  Weil)
* librados: handle reply race with pool deletion (#10372 Sage Weil)
* librbd: cap memory utilization for read requests (Jason Dillaman)
* librbd: do not close a closed parent image on failure (#10030 Jason
  Dillaman)
* librbd: fix diff tests (#10002 Josh Durgin)
* librbd: protect list_children from invalid pools (#10123 Jason Dillaman)
* make check improvemens (Loic Dachary)
* mds: fix ctime updates (#9514 Greg Farnum)
* mds: fix journal import tool (#10025 John Spray)
* mds: fix rare NULL deref in cap flush handler (Greg Farnum)
* mds: handle unknown lock messages (Yan, Zheng)
* mds: store backtrace for straydir (Yan, Zheng)
* mon: abort startup if disk is full (#9502 Joao Eduardo Luis)
* mon: add paxos instrumentation (Sage Weil)
* mon: fix double-free in rare OSD startup path (Sage Weil)
* mon: fix osdmap trimming (#9987 Sage Weil)
* mon: fix paxos corner cases (#9301 #9053 Sage Weil)
* osd: cancel callback on blacklisted watchers (#8315 Samuel Just)
* osd: cleanly abort set-alloc-hint operations during upgrade (#9419 David
  Zafman)
* osd: clear rollback PG metadata on PG deletion (#9293 Samuel Just)
* osd: do not abort deep scrub if hinfo is missing (#10018 Loic Dachary)
* osd: erasure-code regression tests (Loic Dachary)
* osd: fix distro metadata reporting for SUSE (#8654 Danny Al-Gaaf)
* osd: fix full OSD checks during backfill (#9574 Samuel Just)
* osd: fix ioprio parsing (#9677 Loic Dachary)
* osd: fix journal direct-io shutdown (#9073 Mark Kirkwood, Ma Jianpeng,
  Somnath Roy)
* osd: fix journal dump (Ma Jianpeng)
* osd: fix occasional stall during peering or activation (Sage Weil)
* osd: fix past_interval display bug (#9752 Loic Dachary)
* osd: fix rare crash triggered by admin socket dump_ops_in_filght (#9916
  Dong Lei)
* osd: fix snap trimming performance issues (#9487 #9113 Samuel Just, Sage
  Weil, Dan van der Ster, Florian Haas)
* osd: fix snapdir handling on cache eviction (#8629 Sage Weil)
* osd: handle map gaps in map advance code (Sage Weil)
* osd: handle undefined CRUSH results in interval check (#9718 Samuel
  Just)
* osd: include shard in JSON dump of ghobject (#10063 Loic Dachary)
* osd: make backfill reservation denial handling more robust (#9626 Samuel
  Just)
* osd: make misdirected op checks handle EC + primary affinity (#9835
  Samuel Just, Sage Weil)
* osd: mount XFS with inode64 by default (Sage Weil)
* osd: other misc bugs (#9821 #9875 Samuel Just)
* rgw: add .log to default log path (#9353 Alexandre Marangone)
* rgw: clean up fcgi request context (#10194 Yehuda Sadeh)
* rgw: convet header underscores to dashes (#9206 Yehuda Sadeh)
* rgw: copy object data if copy target is in different pool (#9039 Yehuda
  Sadeh)
* rgw: don't try to authenticate CORS peflight request (#8718 Robert
  Hubbard, Yehuda Sadeh)
* rgw: fix civetweb URL decoding (#8621 Yehuda Sadeh)
* rgw: fix hash calculation during PUT (Yehuda Sadeh)
* rgw: fix misc bugs (#9089 #9201 Yehuda Sadeh)
* rgw: fix object tail test (#9226 Sylvain Munaut, Yehuda Sadeh)
* rgw: make sysvinit script run rgw under systemd context as needed
  (#10125 Loic Dachary)
* rgw: separate civetweb log from rgw log (Yehuda Sadeh)
* rgw: set length for keystone token validations (#7796 Mark Kirkwood,
  Yehuda Sadeh)
* rgw: subuser creation fixes (#8587 Yehuda Sadeh)
* rpm: misc packaging improvements (Sandon Van Ness, Dan Mick, Erik
  Logthenberg, Boris Ranto)
* rpm: use standard udev rules for CentOS7/RHEL7 (#9747 Loic Dachary)

Getting Ceph
------------

* Git at git://github.com/ceph/ceph.git
* Tarball at http://ceph.com/download/ceph-0.80.8.tar.gz
* For packages, see http://ceph.com/docs/master/install/get-packages
* For ceph-deploy, see http://ceph.com/docs/master/install/install-ceph-deploy
_______________________________________________
Ceph-maintainers mailing list
<email address hidden>
http://lists.ceph.com/listinfo.cgi/ceph-maintainers-ceph.com

James Page (james-page) on 2015-01-23
Changed in ceph (Ubuntu):
importance: Undecided → High
Changed in ceph (Ubuntu Utopic):
importance: Undecided → High
Changed in ceph (Ubuntu Trusty):
importance: Undecided → High
Changed in ceph (Ubuntu Vivid):
status: New → Invalid
Changed in ceph (Ubuntu Utopic):
status: New → In Progress
Changed in ceph (Ubuntu Trusty):
status: New → In Progress
James Page (james-page) wrote :
Changed in ceph (Ubuntu Trusty):
assignee: nobody → James Page (james-page)
Changed in ceph (Ubuntu Utopic):
assignee: nobody → James Page (james-page)

Hello James, or anyone else affected,

Accepted ceph into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/0.80.8-0ubuntu0.14.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in ceph (Ubuntu Utopic):
status: In Progress → Fix Committed
tags: added: verification-needed
Brian Murray (brian-murray) wrote :

Hello James, or anyone else affected,

Accepted ceph into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/0.80.8-0ubuntu0.14.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in ceph (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: upgrade-software-version

Bug 1420647 is waiting for this SRU to clear -proposed. Just noting this here so I can follow the trail when this bug is marked Fix Released.

James Page (james-page) wrote :

Although this test OK, a performance regression was reported upstream:

  http://tracker.ceph.com/issues/10956

so we don't want to release this just yet.

James Page (james-page) wrote :
Download full text (4.2 KiB)

This is a bugfix release for firefly. It fixes a performance regression
in librbd, an important CRUSH misbehavior (see below), and several RGW
bugs. We have also backported support for flock/fcntl locks to ceph-fuse
and libcephfs.

We recommend that all Firefly users upgrade.

For more detailed information, see
  http://docs.ceph.com/docs/master/_downloads/v0.80.9.txt

Adjusting CRUSH maps
--------------------

* This point release fixes several issues with CRUSH that trigger
  excessive data migration when adjusting OSD weights. These are most
  obvious when a very small weight change (e.g., a change from 0 to
  .01) triggers a large amount of movement, but the same set of bugs
  can also lead to excessive (though less noticeable) movement in
  other cases.

  However, because the bug may already have affected your cluster,
  fixing it may trigger movement *back* to the more correct location.
  For this reason, you must manually opt-in to the fixed behavior.

  In order to set the new tunable to correct the behavior::

     ceph osd crush set-tunable straw_calc_version 1

  Note that this change will have no immediate effect. However, from
  this point forward, any 'straw' bucket in your CRUSH map that is
  adjusted will get non-buggy internal weights, and that transition
  may trigger some rebalancing.

  You can estimate how much rebalancing will eventually be necessary
  on your cluster with::

     ceph osd getcrushmap -o /tmp/cm
     crushtool -i /tmp/cm --num-rep 3 --test --show-mappings > /tmp/a 2>&1
     crushtool -i /tmp/cm --set-straw-calc-version 1 -o /tmp/cm2
     crushtool -i /tmp/cm2 --reweight -o /tmp/cm2
     crushtool -i /tmp/cm2 --num-rep 3 --test --show-mappings > /tmp/b 2>&1
     wc -l /tmp/a # num total mappings
     diff -u /tmp/a /tmp/b | grep -c ^+ # num changed mappings

   Divide the total number of lines in /tmp/a with the number of lines
   changed. We've found that most clusters are under 10%.

   You can force all of this rebalancing to happen at once with::

     ceph osd crush reweight-all

   Otherwise, it will happen at some unknown point in the future when
   CRUSH weights are next adjusted.

Notable Changes
---------------

* ceph-fuse: flock, fcntl lock support (Yan, Zheng, Greg Farnum)
* crush: fix straw bucket weight calculation, add straw_calc_version
  tunable (#10095 Sage Weil)
* crush: fix tree bucket (Rongzu Zhu)
* crush: fix underflow of tree weights (Loic Dachary, Sage Weil)
* crushtool: add --reweight (Sage Weil)
* librbd: complete pending operations before losing image (#10299 Jason
  Dillaman)
* librbd: fix read caching performance regression (#9854 Jason Dillaman)
* librbd: gracefully handle deleted/renamed pools (#10270 Jason Dillaman)
* mon: fix dump of chooseleaf_vary_r tunable (Sage Weil)
* osd: fix PG ref leak in snaptrimmer on peering (#10421 Kefu Chai)
* osd: handle no-op write with snapshot (#10262 Sage Weil)
* radosgw-admin: create subuser when creating user (#10103 Yehuda Sadeh)
* rgw: change multipart uplaod id magic (#10271 Georgio Dimitrakakis,
  Yehuda Sadeh)
* rgw: don't overwrite bucket/object owner when setting ACLs (#10978
  Yehuda Sad...

Read more...

summary: - [SRU] 0.80.8 point release
+ [SRU] 0.80.9 point release

Hello James, or anyone else affected,

Accepted ceph into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/0.80.9-0ubuntu0.14.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Brian Murray (brian-murray) wrote :

Hello James, or anyone else affected,

Accepted ceph into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/0.80.9-0ubuntu0.14.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

James Page (james-page) wrote :

Verfied alongside the recent icehouse 2014.1 point release.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ceph - 0.80.9-0ubuntu0.14.04.1

---------------
ceph (0.80.9-0ubuntu0.14.04.1) trusty; urgency=medium

  [ James Page ]
  * New upstream stable point release (LP: #1413917).

  [ Dave Chiluk ]
  * Increase file limit to prevent hitting the file limit on large
    installations (LP: #1420647)
 -- James Page <email address hidden> Wed, 11 Mar 2015 09:14:35 +0000

Changed in ceph (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for ceph has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ceph - 0.80.9-0ubuntu0.14.10.1

---------------
ceph (0.80.9-0ubuntu0.14.10.1) utopic; urgency=medium

  [ James Page ]
  * New upstream stable point release (LP: #1413917).

  [ Dave Chiluk ]
  * Increase file limit to prevent hitting the file limit on large
    installations (LP: #1420647)
 -- James Page <email address hidden> Wed, 11 Mar 2015 09:16:00 +0000

Changed in ceph (Ubuntu Utopic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers