overlay: permission regression in 5.4.0-51.56 due to patches related to CVE-2020-16120

Bug #1900141 reported by Philipp Wendler
28
This bug affects 5 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Undecided
Seth Forshee
Focal
High
Seth Forshee
Groovy
High
Seth Forshee

Bug Description

SRU Justification

[Impact]

The backports to fix CVE-2020-16120 introduced a regression for overlay mounts within user namespaces. Files with ownership outside of the user namespace can no longer be accessed, even if allowed by both DAC and MAC.

This issue is fixed by the following upstream commit:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b6650dab404c701d7fe08a108b746542a934da84

This commit relaxes the check to remove O_NOATIME from the open flags for the file in the lower filesystem when the overlay filesystem mounter is not privileged with respect to the underlying inode, rather than failing the open as happens now.

[Test Case]

The attached lp1900141.sh script reproduces the issue.

[Where problems could occur]

For the most part this patch restores previous behavior of allowing access to these files while keeping the enhanced permission checks towards the lower filesystem to help prevent unauthorized access to file data in the lower filesystem. The one difference in behavior is that files in the lower filesystem may no longer be opened with the O_NOATIME flag, potentially causing atime updates for these files which were not happening before. If any software expects O_NOATIME behavior in this situation then it could cause problems for that software. However, the correct behavior is that only the inode owner or a process with CAP_FOWNER towards the inode owner is allowed to open with O_NOATIME (as documented in open(2)).

---

We use unprivileged user namespaces with overlay mounts for containers. After recently upgrading our Focal kernels to 5.4.0-51.56 this breaks, one cannot access files through the overlay mount in the container anymore. This is very likely caused by some of the patches that were added in relation to CVE-2020-16120.

The following commands allow to reproduce the problem when executed as an arbitrary non-root user:

mkdir /tmp/test /tmp/test/upper /tmp/test/work /tmp/test/usr
unshare -m -U -r /bin/sh -c "mount -t overlay none /tmp/test/usr -o lowerdir=/usr,upperdir=/tmp/test/upper,workdir=/tmp/test/work; ls -l /tmp/test/usr/bin/id; file /tmp/test/usr/bin/id; /tmp/test/usr/bin/id"

The output when broken is this:

-rwxr-xr-x 1 nobody nogroup 47480 Sep 5 2019 /tmp/test/usr/bin/id
/tmp/test/usr/bin/id: executable, regular file, no read permission
/bin/sh: 1: /tmp/test/usr/bin/id: Operation not permitted

The expected output is this:

-rwxr-xr-x 1 nobody nogroup 43224 Jan 18 2018 /tmp/test/usr/bin/id
/tmp/test/usr/bin/id: ELF 64-bit LSB shared object, ...
uid=0(root) gid=0(root) groups=0(root),65534(nogroup)

These commands create a user namespace and within it mount an overlay of /usr to /tmp/test/usr and then try to access something in it.

This works on Ubuntu Bionic with kernel 4.15.0-121.123 (note that this already includes a fix for CVE-2020-16120) and on kernel 5.4.0-48.52 but is broken on kernel 5.4.0-51.56, no matter whether on Bionic or Focal.

So I strongly suspect that not the actual security fixes for CVE-2020-16120 are the cause, but one of the following two patches that according to the changelogs were applied in the same revision but only to 5.4, not to 4.15:

ovl: call secutiry hook in ovl_real_ioctl()
ovl: check permission to open real file

The mail with the announcement (https://www.openwall.com/lists/oss-security/2020/10/13/6) lists these two commits as separate from the actual security fixes ("may be desired or necessary").

Is it possible to revert these two changes or fix them such that our unprivileged containers work again on Ubuntu kernel 5.4? Or is there a workaround that I can add to my container solution such that this use case works again?

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: linux-image-5.4.0-51-generic 5.4.0-51.56
ProcVersionSignature: User Name 5.4.0-51.56-generic 5.4.65
Uname: Linux 5.4.0-51-generic x86_64
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Oct 14 04:48 seq
 crw-rw---- 1 root audio 116, 33 Oct 14 04:48 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu27.9
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: [Errno 2] No such file or directory: 'fuser'
CasperMD5CheckResult: skip
CurrentDmesg: Error: command ['dmesg'] failed with exit code 1: dmesg: read kernel buffer failed: Operation not permitted
Date: Fri Oct 16 13:02:32 2020
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
 Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU USB Tablet
 Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Lsusb-t:
 /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
     |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
MachineType: QEMU Standard PC (i440FX + PIIX, 1996)
PciMultimedia:

ProcEnviron:
 TERM=screen-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB: 0 bochs-drmdrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.4.0-51-generic root=PARTUUID=59ea2f51-599c-49f2-b9b3-77197e333865 ro console=tty1 console=ttyS0
RelatedPackageVersions:
 linux-restricted-modules-5.4.0-51-generic N/A
 linux-backports-modules-5.4.0-51-generic N/A
 linux-firmware 1.187.3
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.vendor: SeaBIOS
dmi.bios.version: rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-i440fx-5.0
dmi.modalias: dmi:bvnSeaBIOS:bvrrel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org:bd04/01/2014:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-5.0:cvnQEMU:ct1:cvrpc-i440fx-5.0:
dmi.product.name: Standard PC (i440FX + PIIX, 1996)
dmi.product.version: pc-i440fx-5.0
dmi.sys.vendor: QEMU

CVE References

Revision history for this message
Philipp Wendler (philw85) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux (Ubuntu):
status: New → Confirmed
Revision history for this message
Seth Forshee (sforshee) wrote :

I think I see what the problem is, one of the patches adds a check that is probably unnecessary and too restrictive. This is an upstream issue though, so I'm going to follow up with the upstream developers to ensure there isn't a good reason for the check that isn't apparent to me.

Changed in linux (Ubuntu):
assignee: nobody → Seth Forshee (sforshee)
status: Confirmed → In Progress
Revision history for this message
Philipp Wendler (philw85) wrote :

I noticed that the changelog of the kernel package 5.4.0-50.55~18.04.1 for Bionic now also includes the two additional patches, and indeed I can confirm that on Bionic with kernel 5.4.0-54-generic the regression was now also introduced.

Is there an update whether it will be possible to solve this regression? It breaks our container runtime unfortunately.

Revision history for this message
Lane Roberts (lanecroberts) wrote :

This also breaks some of our containers - is there any kind of work-around we can use?

Revision history for this message
Seth Forshee (sforshee) wrote :

Apologies for the delay on this bug. There is a fix upstream in 5.11-rc1, I've backported the fix to the test kernel located here:

https://people.canonical.com/~sforshee/lp1900141/linux-5.4.0-59.65+lp1900141v202101061102/

I'm also attaching a script which reproduces the bug. In my testing the problem looks to be fixed by this patch, but additional testing is appreciated.

Seth Forshee (sforshee)
Changed in linux (Ubuntu Focal):
assignee: nobody → Seth Forshee (sforshee)
importance: Undecided → High
status: New → In Progress
Changed in linux (Ubuntu Groovy):
assignee: nobody → Seth Forshee (sforshee)
importance: Undecided → High
status: New → In Progress
Seth Forshee (sforshee)
description: updated
Revision history for this message
Lane Roberts (lanecroberts) wrote :

Thanks Seth - that appears to fix our problem as well!

Changed in linux (Ubuntu Focal):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Groovy):
status: In Progress → Fix Committed
Seth Forshee (sforshee)
Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-groovy' to 'verification-done-groovy'. If the problem still exists, change the tag 'verification-needed-groovy' to 'verification-failed-groovy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-groovy
tags: added: verification-needed-focal
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Revision history for this message
Philipp Wendler (philw85) wrote :

Thanks!

I tested it on a Focal machine and the -proposed kernel works. However, I don't have a Groovy machine here, is it necessary for me to test this?

I noticed that in the list of affected packages in the bug metadata Bionic is not mentioned. Will the fix also be backported there?

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Seth Forshee (sforshee) wrote : Re: [Bug 1900141] Re: overlay: permission regression in 5.4.0-51.56 due to patches related to CVE-2020-16120

On Mon, Jan 11, 2021 at 11:12:35AM -0000, Philipp Wendler wrote:
> I tested it on a Focal machine and the -proposed kernel works. However,
> I don't have a Groovy machine here, is it necessary for me to test this?

I can verify the fix in groovy.

> I noticed that in the list of affected packages in the bug metadata
> Bionic is not mentioned. Will the fix also be backported there?

It depends on which kernel you are talking about. The bionic GA kernel
(4.15) was not affected based on my testing. If you are seeing problems
with it, please let me know.

The bionic HWE kernel is derived from the kernel source in focal, so
that kernel does not need to be fixed separately from the focal kernel.

Revision history for this message
Seth Forshee (sforshee) wrote :

Confirmed that the attached test script reproduces the problem with 5.8.0-36-generic from groovy-updates. ith 5.8.0-37-generic from groovy-proposed the problem is fixed.

tags: added: verification-done-groovy
removed: verification-needed-groovy
Revision history for this message
Philipp Wendler (philw85) wrote :

Thanks!

>> I noticed that in the list of affected packages in the bug metadata
>> Bionic is not mentioned. Will the fix also be backported there?
>
> It depends on which kernel you are talking about. The bionic GA kernel
> (4.15) was not affected based on my testing. If you are seeing problems
> with it, please let me know.

4.15 was not affected indeed.

> The bionic HWE kernel is derived from the kernel source in focal, so
> that kernel does not need to be fixed separately from the focal kernel.

Ok, just wanted to make sure this is the case.

Everything is fine for me now.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 5.4.0-65.73

---------------
linux (5.4.0-65.73) focal; urgency=medium

  * focal/linux: 5.4.0-65.73 -proposed tracker (LP: #1912220)

  * initramfs unpacking failed (LP: #1835660)
    - SAUCE: lib/decompress_unlz4.c: correctly handle zero-padding around initrds.

  * overlay: permission regression in 5.4.0-51.56 due to patches related to
    CVE-2020-16120 (LP: #1900141)
    - ovl: do not fail because of O_NOATIME

  * Focal update: v5.4.79 upstream stable release (LP: #1907151)
    - net/mlx5: Use async EQ setup cleanup helpers for multiple EQs
    - net/mlx5: poll cmd EQ in case of command timeout
    - net/mlx5: Fix a race when moving command interface to events mode
    - net/mlx5: Add retry mechanism to the command entry index allocation

  * Kernel 5.4.0-56 Wi-Fi does not connect (LP: #1906770)
    - mt76: fix fix ampdu locking

  * [Ubuntu 21.04 FEAT] mpt3sas: Request to include the patch set which supports
    topology where zoning is enabled in expander (LP: #1899802)
    - scsi: mpt3sas: Define hba_port structure
    - scsi: mpt3sas: Allocate memory for hba_port objects
    - scsi: mpt3sas: Rearrange _scsih_mark_responding_sas_device()
    - scsi: mpt3sas: Update hba_port's sas_address & phy_mask
    - scsi: mpt3sas: Get device objects using sas_address & portID
    - scsi: mpt3sas: Rename transport_del_phy_from_an_existing_port()
    - scsi: mpt3sas: Get sas_device objects using device's rphy
    - scsi: mpt3sas: Update hba_port objects after host reset
    - scsi: mpt3sas: Set valid PhysicalPort in SMPPassThrough
    - scsi: mpt3sas: Handling HBA vSES device
    - scsi: mpt3sas: Add bypass_dirty_port_flag parameter
    - scsi: mpt3sas: Handle vSES vphy object during HBA reset
    - scsi: mpt3sas: Add module parameter multipath_on_hba
    - scsi: mpt3sas: Bump driver version to 35.101.00.00

 -- Kleber Sacilotto de Souza <email address hidden> Mon, 18 Jan 2021 17:31:23 +0100

Changed in linux (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 5.8.0-41.46

---------------
linux (5.8.0-41.46) groovy; urgency=medium

  * groovy/linux: 5.8.0-41.46 -proposed tracker (LP: #1912219)

  * Groovy update: upstream stable patchset 2020-12-17 (LP: #1908555) // nvme
    drive fails after some time (LP: #1910866)
    - Revert "nvme-pci: remove last_sq_tail"

  * initramfs unpacking failed (LP: #1835660)
    - SAUCE: lib/decompress_unlz4.c: correctly handle zero-padding around initrds.

  * overlay: permission regression in 5.4.0-51.56 due to patches related to
    CVE-2020-16120 (LP: #1900141)
    - ovl: do not fail because of O_NOATIME

 -- Kleber Sacilotto de Souza <email address hidden> Mon, 18 Jan 2021 17:01:08 +0100

Changed in linux (Ubuntu Groovy):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (20.7 KiB)

This bug was fixed in the package linux - 5.10.0-14.15

---------------
linux (5.10.0-14.15) hirsute; urgency=medium

  * hirsute/linux: 5.10.0-14.15 -proposed tracker (LP: #1913724)

  * Restore palm ejection on multi-input devices (LP: #1913520)
    - HID: multitouch: Apply MT_QUIRK_CONFIDENCE quirk for multi-input devices

  * intel-hid is not loaded on new Intel platform (LP: #1907160)
    - platform/x86: intel-hid: add Rocket Lake ACPI device ID

  * Hirsute update: v5.10.11 upstream stable release (LP: #1913430)
    - scsi: target: tcmu: Fix use-after-free of se_cmd->priv
    - mtd: rawnand: gpmi: fix dst bit offset when extracting raw payload
    - mtd: rawnand: nandsim: Fix the logic when selecting Hamming soft ECC engine
    - i2c: tegra: Wait for config load atomically while in ISR
    - i2c: bpmp-tegra: Ignore unknown I2C_M flags
    - platform/x86: ideapad-laptop: Disable touchpad_switch for ELAN0634
    - ALSA: seq: oss: Fix missing error check in snd_seq_oss_synth_make_info()
    - ALSA: hda/realtek - Limit int mic boost on Acer Aspire E5-575T
    - ALSA: hda/via: Add minimum mute flag
    - crypto: xor - Fix divide error in do_xor_speed()
    - dm crypt: fix copy and paste bug in crypt_alloc_req_aead
    - ACPI: scan: Make acpi_bus_get_device() clear return pointer on error
    - btrfs: don't get an EINTR during drop_snapshot for reloc
    - btrfs: do not double free backref nodes on error
    - btrfs: fix lockdep splat in btrfs_recover_relocation
    - btrfs: don't clear ret in btrfs_start_dirty_block_groups
    - btrfs: send: fix invalid clone operations when cloning from the same file
      and root
    - fs: fix lazytime expiration handling in __writeback_single_inode()
    - pinctrl: ingenic: Fix JZ4760 support
    - mmc: core: don't initialize block size from ext_csd if not present
    - mmc: sdhci-of-dwcmshc: fix rpmb access
    - mmc: sdhci-xenon: fix 1.8v regulator stabilization
    - mmc: sdhci-brcmstb: Fix mmc timeout errors on S5 suspend
    - dm: avoid filesystem lookup in dm_get_dev_t()
    - dm integrity: fix a crash if "recalculate" used without "internal_hash"
    - dm integrity: conditionally disable "recalculate" feature
    - drm/atomic: put state on error path
    - drm/syncobj: Fix use-after-free
    - drm/amdgpu: remove gpu info firmware of green sardine
    - drm/amd/display: DCN2X Find Secondary Pipe properly in MPO + ODM Case
    - drm/i915/gt: Prevent use of engine->wa_ctx after error
    - drm/i915: Check for rq->hwsp validity after acquiring RCU lock
    - ASoC: Intel: haswell: Add missing pm_ops
    - ASoC: rt711: mutex between calibration and power state changes
    - SUNRPC: Handle TCP socket sends with kernel_sendpage() again
    - HID: sony: select CONFIG_CRC32
    - dm integrity: select CRYPTO_SKCIPHER
    - x86/hyperv: Fix kexec panic/hang issues
    - scsi: ufs: Relax the condition of UFSHCI_QUIRK_SKIP_MANUAL_WB_FLUSH_CTRL
    - scsi: ufs: Correct the LUN used in eh_device_reset_handler() callback
    - scsi: qedi: Correct max length of CHAP secret
    - scsi: scsi_debug: Fix memleak in scsi_debug_init()
    - scsi: sd: Suppress spurious errors when WRITE SAME is being disabled
    - riscv: ...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers