shiftfs: allow overlayfs

Bug #1838677 reported by Christian Brauner
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
Undecided
Christian Brauner
Disco
Fix Released
Medium
Unassigned

Bug Description

SRU Justification

Impact: Currently it is not possible to use overlayfs on top of shiftfs. This means Docker inside of LXD cannot make user of the overlay2 graph driver which is blocking users such as Travis from making use of it efficiently.

Regression Potential: Limited to shiftfs and overlayfs on top of shiftfs. Overlayfs does prevent "remote" filesystems such as ceph, nfs, etc. from being used as the underlay. With this patch shiftfs however can be used as an underlay and we special case it as a suitable filesystem to be used under overlayfs. I verified that the patch does not lead to regression on overlayfs workloads that do not make use of shiftfs as underlay. Additionally, I tested Docker with the overlay2 graphdriver on top of shiftfs. This also has not lead to any regressions.

Test case: Building a kernel with the patch:
sudo snap install lxd
sudo lxd init
sudo lxc launch images:ubuntu/bionic b1
sudo lxc config set b1 security.nesting true
sudo lxc restart --force b1
sudo lxc shell b1
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
curl -fsSL get.docker.com | CHANNEL=test sh

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io

sudo systemctl stop docker

cat <<EOF /etc/docker/daemon.json
{
  "storage-driver": "overlay2"
}
EOF

sudo systemctl start docker

docker run -it ubuntu bash

and observe that it works.

Target kernels: All LTS kernels that do support shiftfs, if possible.

Changed in linux (Ubuntu):
assignee: nobody → Christian Brauner (cbrauner)
status: New → Confirmed
status: Confirmed → In Progress
Revision history for this message
Christian Brauner (cbrauner) wrote :
tags: added: shiftfs
Seth Forshee (sforshee)
Changed in linux (Ubuntu):
status: In Progress → Fix Committed
Stefan Bader (smb)
Changed in linux (Ubuntu Disco):
importance: Undecided → Medium
Changed in linux (Ubuntu Disco):
status: New → In Progress
Changed in linux (Ubuntu Disco):
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-disco' to 'verification-done-disco'. If the problem still exists, change the tag 'verification-needed-disco' to 'verification-failed-disco'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-disco
tags: added: verification-done-disco
removed: verification-needed-disco
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-bionic
tags: added: verification-done-bionic
removed: verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (27.8 KiB)

This bug was fixed in the package linux - 5.2.0-13.14

---------------
linux (5.2.0-13.14) eoan; urgency=medium

  * eoan/linux: 5.2.0-13.14 -proposed tracker (LP: #1840261)

  * NULL pointer dereference when Inserting the VIMC module (LP: #1840028)
    - media: vimc: fix component match compare

  * Miscellaneous upstream changes
    - selftests/bpf: remove bpf_util.h from BPF C progs

linux (5.2.0-12.13) eoan; urgency=medium

  * eoan/linux: 5.2.0-12.13 -proposed tracker (LP: #1840184)

  * Eoan update: v5.2.8 upstream stable release (LP: #1840178)
    - scsi: fcoe: Embed fc_rport_priv in fcoe_rport structure
    - libnvdimm/bus: Prepare the nd_ioctl() path to be re-entrant
    - libnvdimm/bus: Fix wait_nvdimm_bus_probe_idle() ABBA deadlock
    - ALSA: usb-audio: Sanity checks for each pipe and EP types
    - ALSA: usb-audio: Fix gpf in snd_usb_pipe_sanity_check
    - HID: wacom: fix bit shift for Cintiq Companion 2
    - HID: Add quirk for HP X1200 PIXART OEM mouse
    - atm: iphase: Fix Spectre v1 vulnerability
    - bnx2x: Disable multi-cos feature.
    - drivers/net/ethernet/marvell/mvmdio.c: Fix non OF case
    - ife: error out when nla attributes are empty
    - ip6_gre: reload ipv6h in prepare_ip6gre_xmit_ipv6
    - ip6_tunnel: fix possible use-after-free on xmit
    - ipip: validate header length in ipip_tunnel_xmit
    - mlxsw: spectrum: Fix error path in mlxsw_sp_module_init()
    - mvpp2: fix panic on module removal
    - mvpp2: refactor MTU change code
    - net: bridge: delete local fdb on device init failure
    - net: bridge: mcast: don't delete permanent entries when fast leave is
      enabled
    - net: bridge: move default pvid init/deinit to NETDEV_REGISTER/UNREGISTER
    - net: fix ifindex collision during namespace removal
    - net/mlx5e: always initialize frag->last_in_page
    - net/mlx5: Use reversed order when unregister devices
    - net: phy: fixed_phy: print gpio error only if gpio node is present
    - net: phylink: don't start and stop SGMII PHYs in SFP modules twice
    - net: phylink: Fix flow control for fixed-link
    - net: phy: mscc: initialize stats array
    - net: qualcomm: rmnet: Fix incorrect UL checksum offload logic
    - net: sched: Fix a possible null-pointer dereference in dequeue_func()
    - net sched: update vlan action for batched events operations
    - net: sched: use temporary variable for actions indexes
    - net/smc: do not schedule tx_work in SMC_CLOSED state
    - net: stmmac: Use netif_tx_napi_add() for TX polling function
    - NFC: nfcmrvl: fix gpio-handling regression
    - ocelot: Cancel delayed work before wq destruction
    - tipc: compat: allow tipc commands without arguments
    - tipc: fix unitilized skb list crash
    - tun: mark small packets as owned by the tap sock
    - net/mlx5: Fix modify_cq_in alignment
    - net/mlx5e: Prevent encap flow counter update async to user query
    - r8169: don't use MSI before RTL8168d
    - bpf: fix XDP vlan selftests test_xdp_vlan.sh
    - selftests/bpf: add wrapper scripts for test_xdp_vlan.sh
    - selftests/bpf: reduce time to execute test_xdp_vlan.sh
    - net: fix bpf_xdp_adjust_head regression for generic-XDP
    - hv_sock: Fi...

Changed in linux (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (34.4 KiB)

This bug was fixed in the package linux - 5.0.0-27.28

---------------
linux (5.0.0-27.28) disco; urgency=medium

  * disco/linux: 5.0.0-27.28 -proposed tracker (LP: #1840816)

  * [Potential Regression] System crashes when running ftrace test in
    ubuntu_kernel_selftests (LP: #1840750)
    - x86/kprobes: Set instruction page as executable

linux (5.0.0-26.27) disco; urgency=medium

  * disco/linux: 5.0.0-26.27 -proposed tracker (LP: #1839972)

  * Packaging resync (LP: #1786013)
    - [Packaging] update helper scripts

  * alsa/hdmi: add icelake hdmi audio support for a Dell machine (LP: #1836916)
    - ALSA: hda: hdmi - add Icelake support
    - ALSA: hda/hdmi - Remove duplicated define
    - ALSA: hda/hdmi - Fix i915 reverse port/pin mapping

  * input/mouse: alps trackpoint-only device doesn't work (LP: #1836752)
    - Input: alps - don't handle ALPS cs19 trackpoint-only device
    - Input: alps - fix a mismatch between a condition check and its comment

  * [18.04 FEAT] Enhanced hardware support (LP: #1836857)
    - s390: report new CPU capabilities
    - s390: add alignment hints to vector load and store

  * System does not auto detect disconnection of external monitor (LP: #1835001)
    - drm/i915: Add support for retrying hotplug
    - drm/i915: Enable hotplug retry

  * [18.04 FEAT] Enhanced CPU-MF hardware counters - kernel part (LP: #1836860)
    - s390/cpum_cf: Add support for CPU-MF SVN 6
    - s390/cpumf: Add extended counter set definitions for model 8561 and 8562

  * EeePC 1005px laptop backlight is off after system boot up (LP: #1837117)
    - platform/x86: asus-wmi: Only Tell EC the OS will handle display hotkeys from
      asus_nb_wmi

  * br_netfilter: namespace sysctl operations (LP: #1836910)
    - netfilter: bridge: port sysctls to use brnf_net
    - netfilter: bridge: namespace bridge netfilter sysctls
    - netfilter: bridge: prevent UAF in brnf_exit_net()

  * ideapad_laptop disables WiFi/BT radios on Lenovo Y540 (LP: #1837136)
    - platform/x86: ideapad-laptop: Remove no_hw_rfkill_list

  * shiftfs: allow overlayfs (LP: #1838677)
    - SAUCE: shiftfs: enable overlayfs on shiftfs

  * bcache: bch_allocator_thread(): hung task timeout (LP: #1784665)
    - bcache: never writeback a discard operation
    - bcache: improve bcache_reboot()
    - SAUCE: bcache: fix deadlock in bcache_allocator

  * Regressions in CMA allocation rework (LP: #1839395)
    - dma-contiguous: do not overwrite align in dma_alloc_contiguous()
    - dma-contiguous: page-align the size in dma_free_contiguous()

  * CVE-2019-3900
    - vhost: introduce vhost_exceeds_weight()
    - vhost_net: fix possible infinite loop
    - vhost: vsock: add weight support
    - vhost: scsi: add weight support

  * Disco update: 5.0.21 upstream stable release (LP: #1837518)
    - bonding/802.3ad: fix slave link initialization transition states
    - cxgb4: offload VLAN flows regardless of VLAN ethtype
    - inet: switch IP ID generator to siphash
    - ipv4/igmp: fix another memory leak in igmpv3_del_delrec()
    - ipv4/igmp: fix build error if !CONFIG_IP_MULTICAST
    - ipv6: Consider sk_bound_dev_if when binding a raw socket to an address
    - ipv6: Fix redi...

Changed in linux (Ubuntu Disco):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.