Xenial: kernel BUG/Oops/crash in fuse_readdir() due to CVE-2020-36322 backport

Bug #1970482 reported by Mauricio Faria de Oliveira
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Invalid
Undecided
Unassigned
Xenial
Fix Released
High
Mauricio Faria de Oliveira

Bug Description

[Impact]

 * Users might hit kernel BUG/Oops/crash with fuse filesystems
   on Xenial kernel 4.4.0-222.255 and later (backport from 4.9),
   including the derivative/optimized kernels (linux-aws below).

 * Introduced by the backport from 4.9 for CVE-2020-36322 [1]
   [1] https://ubuntu.com/security/CVE-2020-36322

 * Offending commit 8deb786162e1 ("fuse: fix bad inode")

   linux-xenial$ git log --oneline origin/master-prep -- fs/fuse/dir.c | head -n1
   8deb786162e1 fuse: fix bad inode

   linux-xenial$ git describe --contains 8deb786162e1
   Ubuntu-4.4.0-222.255~6

[Fix]

  * Check for non-NULL inode pointer before fuse_is_bad(inode)
    in fuse_direntplus_link().

  * (This is the only modified function/patch hunk which seems
    to have issues; all others dereference 'inode' w/out check
    at some point, even before this patch).

[Test Case]

 * Not available at the moment.

[Regression Potential]

 * Probably none, as this changes the hunk/code behavior to
   what it was before the offending patch/backport w/ issue
   was applied (where fuse_is_bad() wasn't called at all if
   inode is NULL), and makes sense with the patch applied;
   also, this same form is used in another hunk, where NULL
   was checked.

[Example Stacktrace]

 kernel: BUG: unable to handle kernel NULL pointer dereference at 00000000000002c0
 kernel: IP: [<ffffffff8132ae16>] fuse_readdir+0x376/0x700
 kernel: PGD 1e3e02c067 PUD 1c8b2aa067 PMD 0
 kernel: Oops: 0000 [#5] SMP
 kernel: Modules linked in: <...>
 kernel: CPU: 1 PID: 12133 Comm: php-fpm Tainted: G D 4.4.0-1138-aws #152-Ubuntu
 kernel: Hardware name: Amazon EC2 m5a.8xlarge/, BIOS 1.0 10/16/2017
 kernel: task: ffff881bcf164600 ti: ffff881bcffec000 task.ti: ffff881bcffec000
 kernel: RIP: 0010:[<ffffffff8132ae16>] [<ffffffff8132ae16>] fuse_readdir+0x376/0x700
 kernel: RSP: 0018:ffff881bcffefe10 EFLAGS: 00010206
 kernel: RAX: ffffc9000524bd00 RBX: 00000000000001a0 RCX: 0000000000000000
 kernel: RDX: 0000000000000001 RSI: ffffc9000524bd00 RDI: ffff881ed25bf3d8
 kernel: RBP: ffff881bcffefea0 R08: 0000000000000000 R09: 0000000000000050
 kernel: R10: ffff881b942a0c68 R11: ffff881ed25bf380 R12: ffff881b942a0bd0
 kernel: R13: ffff880f8ced0d80 R14: ffff881f25cb1800 R15: ffff881ed25bf380
 kernel: FS: 00007f884d100740(0000) GS:ffff880fb8c40000(0000) knlGS:0000000000000000
 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 kernel: CR2: 00000000000002c0 CR3: 0000001cbc963000 CR4: 00000000003406f0
 kernel: Stack:
 kernel: ffff881bcffefef0 0000000000441d7f ffff880fb1c6a000 ffff881b942a0bf8
 kernel: 0000000000000000 ffff881f25cb1800 ffffea006e50a800 ffff881b942a0ca0
 kernel: 00000000ae046100 ffff880fae046100 000000361c41ec1e ffff881b942a0c68
 kernel: Call Trace:
 kernel: [<ffffffff8122d018>] iterate_dir+0x98/0x120
 kernel: [<ffffffff8112f0db>] ? __audit_syscall_entry+0xab/0xf0
 kernel: [<ffffffff8122d589>] SyS_getdents+0x99/0x110
 kernel: [<ffffffff8122d0a0>] ? iterate_dir+0x120/0x120
 kernel: [<ffffffff81848f9b>] entry_SYSCALL_64_fastpath+0x22/0xd0
 kernel: Code: 49 39 80 38 02 00 00 75 12 41 0f b7 00 41 33 44 24 64 f6 c4 f0 0f 84 72 02 00 00 4c 89 ff 4c 89 45 90 e8 ae 65 f0 ff 4c 8b 45 90 <49> 8b 80 c0 02 00 00 4c 89 ff a8 08 0f 85 67 02 00 00 e8 63 5c
 kernel: RIP [<ffffffff8132ae16>] fuse_readdir+0x376/0x700
 kernel: RSP <ffff881bcffefe10>
 kernel: CR2: 00000000000002c0
 kernel: ---[ end trace f89ac23b1e9bb24c ]---

Changed in linux (Ubuntu):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Mauricio Faria de Oliveira (mfo)
Changed in linux (Ubuntu Xenial):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Mauricio Faria de Oliveira (mfo)
Changed in linux (Ubuntu):
status: In Progress → Fix Released
status: Fix Released → Invalid
assignee: Mauricio Faria de Oliveira (mfo) → nobody
importance: High → Undecided
Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :
Download full text (4.0 KiB)

Analysis:
---

Kernel BUG:

 Apr 21 11:39:31 web-12667 kernel: BUG: unable to handle kernel NULL pointer dereference at 00000000000002c0
 Apr 21 11:39:31 web-12667 kernel: IP: [<ffffffff8132ae16>] fuse_readdir+0x376/0x700
 ...
 Apr 21 11:39:31 web-12667 kernel: Oops: 0000 [#5] SMP
 ...
 Apr 21 11:39:31 web-12667 kernel: CPU: 1 PID: 12133 Comm: php-fpm Tainted: G D 4.4.0-1138-aws #152-Ubuntu

(an internal kern.log shows 7 of these, with consistent function+offset and faulting address.)

Faulting source code line:

 $ eu-addr2line -ifae vmlinux-4.4.0-1138-aws fuse_readdir+0x376
 0xffffffff8132ae16

 constant_test_bit inlined at /build/linux-aws-8TZ6DM/linux-aws-4.4.0/fs/fuse/fuse_i.h:699 in fuse_readdir
 /build/linux-aws-8TZ6DM/linux-aws-4.4.0/arch/x86/include/asm/bitops.h:311

 fuse_is_bad
 /build/linux-aws-8TZ6DM/linux-aws-4.4.0/fs/fuse/fuse_i.h:699

 fuse_direntplus_link
 /build/linux-aws-8TZ6DM/linux-aws-4.4.0/fs/fuse/dir.c:1267

 parse_dirplusfile
 /build/linux-aws-8TZ6DM/linux-aws-4.4.0/fs/fuse/dir.c:1342

 fuse_readdir
 /build/linux-aws-8TZ6DM/linux-aws-4.4.0/fs/fuse/dir.c:1392

Source code:

Issue: 'if (!inode)' doesn't prevent 'if (fuse_is_bad(inode)' from running with 'inode == NULL'.

 1194 static int fuse_direntplus_link(struct file *file,
 1195 struct fuse_direntplus *direntplus,
 1196 u64 attr_version)
 1197 {
 ...
 1240 if (dentry) {
 1241 inode = d_inode(dentry);
 1242 if (!inode) {
 1243 d_drop(dentry);
 1244 } else if ...
 1246 ...
 1247 } else if (is_bad_inode(inode)) {
 1248 err = -EIO;
 1249 goto out;
 1250 } else {
 1251 ...
 1266 }
 1267 if (fuse_is_bad(inode)) {
 ...
 1272 }

 697 static inline bool fuse_is_bad(struct inode *inode)
 698 {
 699 return unlikely(test_bit(FUSE_I_BAD, &get_fuse_inode(inode)->state));
 700 }

 681 static inline struct fuse_inode *get_fuse_inode(struct inode *inode)
 682 {
 683 return container_of(inode, struct fuse_inode, inode);
 684 }

Struct field offset (matches faulting address, i.e., NULL + 0x2c0)

 $ pahole --hex -C fuse_inode vmlinux-4.4.0-1138-aws | grep state
  long unsigned int state; /* 0x2c0 0x8 */

The origin of the issue is in the backport to linux-xenial from linux-stable 4.9,
which comes from linux mainline.

- mainline: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5d069dbe8aaf2a197142558b6fb2978189ba3454
- stable49: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/fs/fuse/dir.c?h=v4.9.311&id=3a2f8823aa565cc67bdd00c4cd5e1d8ad81e8436
- backport: https://git.launchpad.net/~canonical-kernel-esm/canonical-kernel-esm/+git/linux-xenial/commit/?h=master-prep&id=8deb786162e1e4cf73ae4c56d62b86f0d7c8ade0

In mainline there's no fuse_direntplus_link()

In stable/4.9, there's an '!inode' check w/ 'goto retry' that skips 'fuse_is_bad()'
- https://git.kernel.org/pub/scm/linux/kernel/gi...

Read more...

Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

[SRU][Xenial][PATCH] UBUNTU: SAUCE: fuse: fix bad !inode in fuse_direntplus_link()

Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

APPLIED: [SRU][Xenial][PATCH] UBUNTU: SAUCE: fuse: fix bad !inode in fuse_direntplus_link()

Changed in linux (Ubuntu Xenial):
status: In Progress → Fix Committed
Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

The fix has been released on Xenial ESM (kernel version 4.4.0-224.257).

$ git log --oneline -1 341e4f5e9e07
341e4f5e9e07 UBUNTU: SAUCE: fuse: fix bad !inode in fuse_direntplus_link()

$ git describe --contains 341e4f5e9e07
Ubuntu-4.4.0-224.257~6

Changed in linux (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.