ceph: fix inode number handling on arches with 32-bit ino_t

Bug #1899582 reported by bugproxy on 2020-10-13
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu on IBM z Systems
High
Skipper Bug Screeners
linux (Ubuntu)
Undecided
Canonical Kernel Team
Focal
Medium
Frank Heimes
Groovy
Undecided
Canonical Kernel Team

Bug Description

SRU Justification:
==================

[Impact]

* A problem occurs with 32-bit ino_t values on 64-bit architectures (like s390x).

* Without this patch, cephfs kernel client does not work properly on s390x: the initial symtom is missing the first created file/directory on cephfs mount point as reported in https://tracker.ceph.com/issues/46828.

[Fix]

* cherry-pick for groovy: ebce3eb2f7ef9f6ef01a60874ebd232450107c9a ebce3eb2f7ef "ceph: fix inode number handling on arches with 32-bit ino_t"

* backport for focal: https://launchpadlibrarian.net/501822049/0001-ceph-fix-inode-number-handling-on-arches-with-32-bit.patch

[Test Case]

* IBM Development team and Test team have carried out simple testing and stress testing by automatically creating many Ceph clusters on s390x (with different configuration each time), doing the cephfs kernel mounting, checking inode numbers, running stress tests. The same procedures were done on x86_64 arch by Test team.

[Regression Potential]

* There is a certain regression risk with this patch, since:

* common ceph filesystem code in fs/ceph/ is modified

* the inode handling is touched

* ceph is used for supplemental volumes

* and as the original author of this patch (Jeff Layton) wrote in the commit message, the potential issue might happen on 32-bit arches only and he also provided workaround for those arches.

[Other]

* The above patch/commit was upstream accepted with kernel 5.9.

* Cherry picking the patch/commit to groovy works fine, hence this SRU.

* For focal a cherry pick does not apply cleanly, hence a backport got added to LP 1899582.

__________

Backport provided for git-commit to 20.04 kernel.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ebce3eb2f7ef9f6ef01a60874ebd232450107c9a

Backport patch attached for commit ebce3eb upstream.

It was applied and tested on top of Ubuntu commit :

linux (5.4.0-48.52) focal; urgency=medium

Default Comment by Bridge

tags: added: architecture-s39064 bugnameltc-187973 severity-high targetmilestone-inin2004
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
affects: ubuntu → ceph (Ubuntu)
Changed in ubuntu-z-systems:
importance: Undecided → High
affects: ceph (Ubuntu) → linux (Ubuntu)
Dimitri John Ledkov (xnox) wrote :

Has this been forwarded to the stable trees?

I do not see it applied on 5.8.x-stable or lower trees.

Currently only available in linux-5.9 which we are not shipping yet.

Changed in linux (Ubuntu):
status: New → Incomplete
Frank Heimes (fheimes) on 2020-10-13
Changed in ubuntu-z-systems:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)
Changed in linux (Ubuntu Groovy):
assignee: Skipper Bug Screeners (skipper-screen-team) → nobody

------- Comment From <email address hidden> 2020-10-13 11:40 EDT-------
Hi,

This patch is not going to be merged to stable 5.8 tree, more details at : https://lkml.org/lkml/2020/9/5/106.

Thus we have to do this backport with minor change for 5.4 tree for Ubuntu 20.04 LTS. The most significant change/difference from the original commit is in the hunk in fs/ceph/inode.c.

Frank Heimes (fheimes) wrote :

If it's not possible (due user-visible changes) to get it marked for stable, I'm wondering if it can be added to groovy -- could become difficult.
On the other hand side if the backport should be applied to focal, it needs to be in groovy first, to avoid regressions on updates.
I verified that the commit cherry-picks cleanly on groovy master-next, hence I'll create a kernel SRU, but this will land (if at all) after 20.10 GA, since we already reached the 20.10 kernel freeze.

Frank Heimes (fheimes) wrote :

I just noticed that it is a user-visible change on 32-bit arches only.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2020-10-14 04:43 EDT-------
This is the original bug report at Ceph project : https://tracker.ceph.com/issues/46828. Might help to clarify a bit ...

Frank Heimes (fheimes) on 2020-10-14
Changed in linux (Ubuntu Groovy):
status: Incomplete → New
Changed in ubuntu-z-systems:
status: New → Triaged
Dimitri John Ledkov (xnox) wrote :

So

>> 1/ inode numbers will be seen to have changed between kernel versions.
>> 32-bit arches will see large inode numbers now instead of the hashed
>> ones they saw before.
>>

That seems fine to me. This is all during runtime only, and doesn't change ondisk format at all right?

>> 2/ any really old software not built with LFS support may start failing
>> stat() calls with -EOVERFLOW on inode numbers >2^32. Nothing much we
>> can do about these, but hopefully the intersection of people running
>> such code on ceph will be very small.

I think we universally support LFS even on armhf & i386 arches. Also note the https://lintian.debian.org/tags/binary-file-built-without-LFS-support.html

It may have been better to scope the patch for backport to s390x only, however, given that hwe kernels will have this eventually, imho it is fine to submit this.

Btw, you can also submit kernel patch to the kernel-team mailing list, if it is formatted as per Ubuntu Kernel team guidelines with like BugLink: https://bugs.launchpad.net/bugs/1899582.

Note we are past kernel freeze now, and there is no SRU cycle between now and 8th of November, due to datacenter move as announced in https://lists.ubuntu.com/archives/kernel-sru-announce/2020-September/000170.html Hence the earlier this can land is probably December.

Frank Heimes (fheimes) on 2020-10-14
description: updated
Changed in linux (Ubuntu Focal):
assignee: nobody → Frank Heimes (fheimes)
Changed in linux (Ubuntu Groovy):
assignee: nobody → Frank Heimes (fheimes)
Frank Heimes (fheimes) on 2020-10-14
description: updated
description: updated
Frank Heimes (fheimes) wrote :

Kernel SRU request submitted for groovy:
https://lists.ubuntu.com/archives/kernel-team/2020-October/thread.html#114060
changing status to 'In Progress' for groovy.

Changed in linux (Ubuntu Groovy):
status: New → In Progress
assignee: Frank Heimes (fheimes) → Canonical Kernel Team (canonical-kernel-team)
Changed in ubuntu-z-systems:
status: Triaged → In Progress
Francis Ginther (fginther) wrote :

@xnox, @fheimes, we are preparing a groovy respin which will pick this up.

Frank Heimes (fheimes) wrote :

@fginther: glad to hear that - thanks a lot!

Frank Heimes (fheimes) wrote :

Kernel test builds based on groovy master-next (cherry-pick) and focal master-next (backport) are available here for further testing: https://people.canonical.com/~fheimes/lp1899582/

Frank Heimes (fheimes) wrote :

Kernel SRU request submitted for focal:
https://lists.ubuntu.com/archives/kernel-team/2020-October/thread.html#114069
changing status to 'In Progress' for focal.

Changed in linux (Ubuntu Focal):
status: New → In Progress
Stefan Bader (smb) on 2020-10-16
Changed in linux (Ubuntu Focal):
importance: Undecided → Medium
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2020-10-19 03:28 EDT-------
(In reply to comment #25)
> Kernel test builds based on groovy master-next (cherry-pick) and focal
> master-next (backport) are available here for further testing:
> https://people.canonical.com/~fheimes/lp1899582/

The kernels seem both fine on groovy and focal, respectively.

Frank Heimes (fheimes) wrote :

That's great - thx for testing, Tuan!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 5.8.0-25.26

---------------
linux (5.8.0-25.26) groovy; urgency=medium

  * groovy/linux: 5.8.0-25.26 -proposed tracker (LP: #1899940)

  * CVE-2020-12351
    - Bluetooth: L2CAP: Fix calling sk_filter on non-socket based channel

  * CVE-2020-12352
    - Bluetooth: A2MP: Fix not initializing all members

  * CVE-2020-12351 // CVE-2020-12352 // CVE-2020-24490
    - Bluetooth: Disable High Speed by default
    - Bluetooth: MGMT: Fix not checking if BT_HS is enabled
    - [Config] Disable BlueZ highspeed support

  * ec2-hibinit-agent needs to properly initialize swap file (LP: #1892728)
    - ext4: implement swap_activate aops using iomap

 -- Andrea Righi <email address hidden> Thu, 15 Oct 2020 12:09:24 +0200

Changed in linux (Ubuntu Groovy):
status: In Progress → Fix Released
Ian (ian-may) 17 hours ago
Changed in linux (Ubuntu Focal):
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.