shiftfs: O_TMPFILE reports ESTALE

[Christian Brauner]

SRU Justification

Impact: Christian Kellner reported that creating temporary files via O_TMPFILE shiftfs reports ESTALE. This can be reproduced via:

import tempfile
import os

def test():
    with tempfile.TemporaryFile() as fd:
        fd.write("data".encode('utf-8'))
        # re-open the file to get a read-only file descriptor
        return open(f"/proc/self/fd/{fd.fileno()}", "r")

def main():
   fd = test()
   fd.close()

if __name__ == "__main__":
    main()

a similar issue was reported here:
https://github.com/systemd/systemd/issues/14861

Fix: Our revalidate methods were very opinionated about whether or not a dentry was valid when we really should've just let the underlay tell us what's what. This has led to bugs where a ESTALE was returned for e.g. temporary files that were created and directly re-opened afterwards through /proc/<pid>/fd/<nr-of-deleted-file>. When a file is re-opened through /proc/<pid>/fd/<nr> LOOKUP_JUMP is set and the vfs will revalidate via d_weak_revalidate(). Since the file has been unhashed or even already gone negative we'd fail the open when we should've succeeded.

I had also foolishly provided a .tmpfile method which so far only has caused us trouble. If we really need this then we can reimplement it properly but I doubt it. Remove it for now.

Regression Potential: Limited to shiftfs.

Test Case: Build a kernel with fix applied and run above reproducer.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-eoan' to 'verification-done-eoan'. If the problem still exists, change the tag 'verification-needed-eoan' to 'verification-failed-eoan'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-focal' to 'verification-done-focal'. If the problem still exists, change the tag 'verification-needed-focal' to 'verification-failed-focal'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 5.4.0-31.35

---------------
linux (5.4.0-31.35) focal; urgency=medium

  * focal/linux: 5.4.0-31.35 -proposed tracker (LP: #1877253)

  * Intermittent display blackouts on event (LP: #1875254)
    - drm/i915: Limit audio CDCLK>=2*BCLK constraint back to GLK only

  * Unable to handle kernel pointer dereference in virtual kernel address space
    on Eoan (LP: #1876645)
    - SAUCE: overlayfs: fix shitfs special-casing

linux (5.4.0-30.34) focal; urgency=medium

  * focal/linux: 5.4.0-30.34 -proposed tracker (LP: #1875385)

  * ubuntu/focal64 fails to mount Vagrant shared folders (LP: #1873506)
    - [Packaging] Move virtualbox modules to linux-modules
    - [Packaging] Remove vbox and zfs modules from generic.inclusion-list

  * linux-image-5.0.0-35-generic breaks checkpointing of container
    (LP: #1857257)
    - SAUCE: overlayfs: use shiftfs hacks only with shiftfs as underlay

  * shiftfs: broken shiftfs nesting (LP: #1872094)
    - SAUCE: shiftfs: record correct creator credentials

  * Add debian/rules targets to compile/run kernel selftests (LP: #1874286)
    - [Packaging] add support to compile/run selftests

  * shiftfs: O_TMPFILE reports ESTALE (LP: #1872757)
    - SAUCE: shiftfs: fix dentry revalidation

  * LIO hanging in iscsit_free_session and iscsit_stop_session (LP: #1871688)
    - scsi: target: iscsi: calling iscsit_stop_session() inside
      iscsit_close_session() has no effect

  * [ICL] TC port in legacy/static mode can't be detected due TCCOLD
    (LP: #1868936)
    - SAUCE: drm/i915: Align power domain names with port names
    - SAUCE: drm/i915/display: Move out code to return the digital_port of the aux
      ch
    - SAUCE: drm/i915/display: Add intel_legacy_aux_to_power_domain()
    - SAUCE: drm/i915/display: Split hsw_power_well_enable() into two
    - SAUCE: drm/i915/tc/icl: Implement TC cold sequences
    - SAUCE: drm/i915/tc: Skip ref held check for TC legacy aux power wells
    - SAUCE: drm/i915/tc/tgl: Implement TC cold sequences
    - SAUCE: drm/i915/tc: Catch TC users accessing FIA registers without enable
      aux
    - SAUCE: drm/i915/tc: Do not warn when aux power well of static TC ports
      timeout

  * alsa/sof: external mic can't be deteced on Lenovo and HP laptops
    (LP: #1872569)
    - SAUCE: ASoC: intel/skl/hda - set autosuspend timeout for hda codecs

  * amdgpu kernel errors in Linux 5.4 (LP: #1871248)
    - drm/amd/display: Stop if retimer is not available

  * Focal update: v5.4.34 upstream stable release (LP: #1874111)
    - amd-xgbe: Use __napi_schedule() in BH context
    - hsr: check protocol version in hsr_newlink()
    - l2tp: Allow management of tunnels and session in user namespace
    - net: dsa: mt7530: fix tagged frames pass-through in VLAN-unaware mode
    - net: ipv4: devinet: Fix crash when add/del multicast IP with autojoin
    - net: ipv6: do not consider routes via gateways for anycast address check
    - net: phy: micrel: use genphy_read_status for KSZ9131
    - net: qrtr: send msgs from local of same id as broadcast
    - net: revert default NAPI poll timeout to 2 jiffies
    - net: tun: record RX queue in skb before do_xdp_gener...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 5.3.0-53.47

---------------
linux (5.3.0-53.47) eoan; urgency=medium

  * eoan/linux: 5.3.0-53.47 -proposed tracker (LP: #1877257)

  * Intermittent display blackouts on event (LP: #1875254)
    - drm/i915: Limit audio CDCLK>=2*BCLK constraint back to GLK only

  * Unable to handle kernel pointer dereference in virtual kernel address space
    on Eoan (LP: #1876645)
    - SAUCE: overlayfs: fix shitfs special-casing

linux (5.3.0-52.46) eoan; urgency=medium

  * eoan/linux: 5.3.0-52.46 -proposed tracker (LP: #1874752)

  * alsa: make the dmic detection align to the mainline kernel-5.6
    (LP: #1871284)
    - ALSA: hda: add Intel DSP configuration / probe code
    - ALSA: hda: fix intel DSP config
    - ALSA: hda: Allow non-Intel device probe gracefully
    - ALSA: hda: More constifications
    - ALSA: hda: Rename back to dmic_detect option
    - [Config] SND_INTEL_DSP_CONFIG=m
    - [packaging] Remove snd-intel-nhlt from modules

  * built-using constraints preventing uploads (LP: #1875601)
    - temporarily drop Built-Using data

  * ubuntu/focal64 fails to mount Vagrant shared folders (LP: #1873506)
    - [Packaging] Move virtualbox modules to linux-modules
    - [Packaging] Remove vbox and zfs modules from generic.inclusion-list

  * linux-image-5.0.0-35-generic breaks checkpointing of container
    (LP: #1857257)
    - SAUCE: overlayfs: use shiftfs hacks only with shiftfs as underlay

  * shiftfs: broken shiftfs nesting (LP: #1872094)
    - SAUCE: shiftfs: record correct creator credentials

  * Add debian/rules targets to compile/run kernel selftests (LP: #1874286)
    - [Packaging] add support to compile/run selftests

  * shiftfs: O_TMPFILE reports ESTALE (LP: #1872757)
    - SAUCE: shiftfs: fix dentry revalidation

  * getitimer returns it_value=0 erroneously (LP: #1349028)
    - [Config] CONTEXT_TRACKING_FORCE policy should be unset

  * 5.3.0-46-generic - i915 - frequent GPU hangs / resets rcs0 (LP: #1872001)
    - drm/i915/execlists: Preempt-to-busy
    - drm/i915/gt: Detect if we miss WaIdleLiteRestore
    - drm/i915/execlists: Always force a context reload when rewinding RING_TAIL

  * alsa/sof: external mic can't be deteced on Lenovo and HP laptops
    (LP: #1872569)
    - SAUCE: ASoC: intel/skl/hda - set autosuspend timeout for hda codecs

  * Eoan update: upstream stable patchset 2020-04-22 (LP: #1874325)
    - ARM: dts: sun8i-a83t-tbs-a711: HM5065 doesn't like such a high voltage
    - bus: sunxi-rsb: Return correct data when mixing 16-bit and 8-bit reads
    - net: vxge: fix wrong __VA_ARGS__ usage
    - hinic: fix a bug of waitting for IO stopped
    - hinic: fix wrong para of wait_for_completion_timeout
    - cxgb4/ptp: pass the sign of offset delta in FW CMD
    - qlcnic: Fix bad kzalloc null test
    - i2c: st: fix missing struct parameter description
    - cpufreq: imx6q: Fixes unwanted cpu overclocking on i.MX6ULL
    - media: venus: hfi_parser: Ignore HEVC encoding for V1
    - firmware: arm_sdei: fix double-lock on hibernate with shared events
    - null_blk: Fix the null_add_dev() error path
    - null_blk: Handle null_add_dev() failures properly
    - null_blk: fix spuri...

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted linux-azure-5.3 (5.3.0-1034.35~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

zfs-linux/0.7.5-1ubuntu16.9 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-azure-5.3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted linux-aws-5.3 (5.3.0-1032.34~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

zfs-linux/0.7.5-1ubuntu16.9 (arm64, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-aws-5.3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted linux-gcp-5.3 (5.3.0-1032.34~18.04.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

zfs-linux/0.7.5-1ubuntu16.9 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#linux-gcp-5.3

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 5.4.0-42.46

---------------
linux (5.4.0-42.46) focal; urgency=medium

  * focal/linux: 5.4.0-42.46 -proposed tracker (LP: #1887069)

  * linux 4.15.0-109-generic network DoS regression vs -108 (LP: #1886668)
    - SAUCE: Revert "netprio_cgroup: Fix unlimited memory leak of v2 cgroups"

linux (5.4.0-41.45) focal; urgency=medium

  * focal/linux: 5.4.0-41.45 -proposed tracker (LP: #1885855)

  * Packaging resync (LP: #1786013)
    - update dkms package versions

  * CVE-2019-19642
    - kernel/relay.c: handle alloc_percpu returning NULL in relay_open

  * CVE-2019-16089
    - SAUCE: nbd_genl_status: null check for nla_nest_start

  * CVE-2020-11935
    - aufs: do not call i_readcount_inc()

  * ip_defrag.sh in net from ubuntu_kernel_selftests failed with 5.0 / 5.3 / 5.4
    kernel (LP: #1826848)
    - selftests: net: ip_defrag: ignore EPERM

  * Update lockdown patches (LP: #1884159)
    - SAUCE: acpi: disallow loading configfs acpi tables when locked down

  * seccomp_bpf fails on powerpc (LP: #1885757)
    - SAUCE: selftests/seccomp: fix ptrace tests on powerpc

  * Introduce the new NVIDIA 418-server and 440-server series, and update the
    current NVIDIA drivers (LP: #1881137)
    - [packaging] add signed modules for the 418-server and the 440-server
      flavours

 -- Khalid Elmously <email address hidden> Thu, 09 Jul 2020 19:50:26 -0400