luminous: bluestore rocksdb max_background_compactions regression in 12.2.13

[James Page]

There is a potential regression in the rocksdb max_background_compactions change in the 12.2.13 point release that is in bionic-proposed.

https://tracker.ceph.com/issues/44937 was raised as the rocksdb version in the bionic ceph source does not contain:

  https://github.com/facebook/rocksdb/pull/3926

which ensures that any overlap between background compactions is detected and handled.

We either need to revert to a single background compaction as the default setting, or pick the fix into the rocksdb source in src:ceph/bionic

[James Page (james-page)]

I'm going to revert the default configuration change - this is a minumal, least risk approach to this issue.

Its possible that we might want to pick the rocksdb fixes as well but that diverges Ubuntu from upstream in a core part of ceph which I'm not 100% comfortable with.

Feedback on the upstream tracker bug indicates there is a lot of evidence that increasing concurrency has not resulted in data loss.

[Łukasz Zemczak (sil2100)]

Hello James, or anyone else affected,

Accepted ceph into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ceph/12.2.13-0ubuntu0.18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[James Page (james-page)]

standard verification of bionic-proposed passed:

======
Totals
======
Ran: 92 tests in 709.8995 sec.
 - Passed: 84
 - Skipped: 8
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 989.9518 sec.

[James Page (james-page)]

xenial-queens/proposed

======
Totals
======
Ran: 92 tests in 707.9748 sec.
 - Passed: 84
 - Skipped: 8
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 897.0776 sec.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ceph - 12.2.13-0ubuntu0.18.04.2

---------------
ceph (12.2.13-0ubuntu0.18.04.2) bionic; urgency=medium

  * d/p/bug1871820.patch: Revert change in default concurrency for
    rocksdb background compactions to avoid potential data loss
    (LP: #1871820).

ceph (12.2.13-0ubuntu0.18.04.1) bionic; urgency=medium

  * New upstream point release (LP: #1861793).
  * d/p/bug1847544.patch,ceph-volume-wait-for-lvs.patch,dont-validate-fs-
    caps-on-authorize.patch,issue37490.patch,issue38454.patch,rgw-gc-use-
    aio.patch: Drop, all included in upstream release.
  * d/p/*: Refresh as needed.
  * d/p/bug1804261.patch: Cherry pick fix to ensure that ceph-volume
    tries and interval environment variables are converted to int
    (LP: #1804261).

 -- James Page <email address hidden> Tue, 19 May 2020 08:40:13 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for ceph has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.