OverlayFS: Wrong mnt_id and path reported in /proc

[Saied Kazemi]

SRU Justification:

[Impact]

I am running Ubuntu Vivid (3.19.0-22-generic #22-Ubuntu SMP Tue Jun 16 17:15:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux). There are two issues in OverlayFS as follows:

 1. /proc/<pid>/fdinfo/<fd> reports wrong mnt_id (non-existent in /proc/<pid>/mountinfo)

 2. /proc/<pid>/fd/<fd> shows incorrect path for the symlink target

[Fix]

These problems can be easily reproduced with the shell script below. Fortunately, both problems have been fixed in the upstream's master branch of the kernel via the following commits:

    torvalds 4.0: 155e35d4d VFS: Introduce inode-getting helpers for layered/unioned fs environments (this is already in ubuntu-vivid)

    torvalds 4.0: df1a085af VFS: Add a fallthrough flag for marking virtual dentries
    torvalds 4.2-rc2: f25801ee4 overlay: Call ovl_drop_write() earlier in ovl_dentry_open()
    torvalds 4.2-rc2: 4bacc9c92 overlayfs: Make f_path always point to the overlay and f_inode to the underlay
    torvalds 4.2-rc2: 9391dd00d fix a braino in ovl_d_select_inode()

(The commit df1a085af may technically not be needed but it prevents a conflict when applying 4bacc9c92.)

[Test Case]

Simple script to reproduce the problem:

#!/bin/bash

set -eu

ERROR=0

setup() {
 setup_mount
 start_python > /dev/null 2>&1 &
 BASH_PID=$!
 PYTHON_PID=$(ps -C python | awk '/python/ { print $1 }')
 sleep 1
}

setup_mount() {
 mkdir overlay_test
 cd overlay_test
 mkdir a b c z
 sudo mount -t overlay -o lowerdir=a,upperdir=b,workdir=c overlayfs z
}

start_python() {
 python << EOF
import time
fd = open("z/file", "w")
time.sleep(10)
EOF
}

check_path() {
 WD=$(pwd)
 if ! ls -l /proc/$PYTHON_PID/fd | grep -qw "$WD/z/file"; then
  ERROR=1
  echo "ERROR! expected $WD/z/file"
  set -x
  ls -l /proc/$PYTHON_PID/fd
  set +x
 fi
}

check_mnt_id() {
 FDINFO_MNT_ID=$(awk '/mnt_id:/ { print $2 }' /proc/$PYTHON_PID/fdinfo/3)
 MOUNTINFO_MNT_ID=$(awk '/overlayfs/ { print $1 }' /proc/self/mountinfo)
 if [[ $FDINFO_MNT_ID -ne $MOUNTINFO_MNT_ID ]]; then
  ERROR=1
  echo "ERROR! mnt_id $FDINFO_MNT_ID not in /proc/self/mountinfo"
  set -x
  cat /proc/$PYTHON_PID/fdinfo/3
  cat /proc/self/mountinfo
  set +x
 fi
}

finish() {
 kill -INT $PYTHON_PID > /dev/null 2>&1
 sudo umount z
 cd $ORIG_WD
 rm -rf overlay_test
}

main() {
 ORIG_WD=$(pwd)
 setup

 check_path
 check_mnt_id

 finish
 [[ $ERROR -eq 0 ]] && echo "OverlayFS looks good."
}

main

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1479468

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Chris J Arges (arges)]

I posted these patches to the kernel team ML. I've tested the above reproducer script and it does indeed fix the issue, and in addition I've ran the unionmount-testsuite 'sudo ./run --ov' on the new kernel and no regressions were noticed (all test passing).

[Saied Kazemi (saied)]

Great! Thanks a lot Chris.

--Saied

On Thu, Jul 30, 2015 at 7:43 AM, Chris J Arges <email address hidden>
wrote:

> I posted these patches to the kernel team ML. I've tested the above
> reproducer script and it does indeed fix the issue, and in addition I've
> ran the unionmount-testsuite 'sudo ./run --ov' on the new kernel and no
> regressions were noticed (all test passing).
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1479468
>
> Title:
> OverlayFS: Wrong mnt_id and path reported in /proc
>
> Status in linux package in Ubuntu:
> Triaged
> Status in linux source package in Vivid:
> Triaged
>
> Bug description:
> SRU Justification:
>
> [Impact]
>
> I am running Ubuntu Vivid (3.19.0-22-generic #22-Ubuntu SMP Tue Jun 16
> 17:15:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux). There are two
> issues in OverlayFS as follows:
>
> 1. /proc/<pid>/fdinfo/<fd> reports wrong mnt_id (non-existent in
> /proc/<pid>/mountinfo)
>
> 2. /proc/<pid>/fd/<fd> shows incorrect path for the symlink target
>
> [Fix]
>
> These problems can be easily reproduced with the shell script below.
> Fortunately, both problems have been fixed in the upstream's master
> branch of the kernel via the following commits:
>
> torvalds 4.0: 155e35d4d VFS: Introduce inode-getting helpers for
> layered/unioned fs environments (this is already in ubuntu-vivid)
>
> torvalds 4.0: df1a085af VFS: Add a fallthrough flag for marking
> virtual dentries
> torvalds 4.2-rc2: f25801ee4 overlay: Call ovl_drop_write() earlier
> in ovl_dentry_open()
> torvalds 4.2-rc2: 4bacc9c92 overlayfs: Make f_path always point to
> the overlay and f_inode to the underlay
> torvalds 4.2-rc2: 9391dd00d fix a braino in ovl_d_select_inode()
>
> (The commit df1a085af may technically not be needed but it prevents a
> conflict when applying 4bacc9c92.)
>
> [Test Case]
>
> Simple script to reproduce the problem:
>
> #!/bin/bash
>
> set -eu
>
> ERROR=0
>
> setup() {
> setup_mount
> start_python > /dev/null 2>&1 &
> BASH_PID=$!
> PYTHON_PID=$(ps -C python | awk '/python/ { print $1 }')
> sleep 1
> }
>
> setup_mount() {
> mkdir overlay_test
> cd overlay_test
> mkdir a b c z
> sudo mount -t overlay -o lowerdir=a,upperdir=b,workdir=c overlayfs z
> }
>
> start_python() {
> python << EOF
> import time
> fd = open("z/file", "w")
> time.sleep(10)
> EOF
> }
>
> check_path() {
> WD=$(pwd)
> if ! ls -l /proc/$PYTHON_PID/fd | grep -qw "$WD/z/file"; then
> ERROR=1
> echo "ERROR! expected $WD/z/file"
> set -x
> ls -l /proc/$PYTHON_PID/fd
> set +x
> fi
> }
>
> check_mnt_id() {
> FDINFO_MNT_ID=$(awk '/mnt_id:/ { print $2 }' /proc/$PYTHON_PID/fdinfo/3)
> MOUNTINFO_MNT_ID=$(awk '/overlayfs/ { print $1 }' /proc/self/mountinfo)
> if [[ $FDINFO_MNT_ID -ne $MOUNTINFO_MNT_ID ]]; then
> ERROR=1
> echo "ERROR! mnt_id $FDINFO_MNT_ID not in /proc/self/mountinfo"
> set -x
> cat /proc/$PYTHON_PID/fdinfo/3
> cat /proc/self/mountinfo
> set +x
> fi
> }
>
> f...

[Luis Henriques (henrix)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-vivid' to 'verification-done-vivid'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Saied Kazemi (saied)]

Installed from -proposed (see below) and verified that it fixes the OverlayFS mnt_id and symlink target path issues.

Selected version '3.19.0-27.29' (Ubuntu:15.04/vivid-proposed [amd64]) for 'linux-image-3.19.0-27-generic'

tags:changed: verification-done-vivid

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 3.19.0-28.30

---------------
linux (3.19.0-28.30) vivid; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1490606

  [ Gary Wang ]

  * SAUCE: i915_bpo: drm/i915: set CDCLK if DPLL0 enabled during resuming
    from S3
    - LP: #1490035

  [ Timo Aaltonen ]

  * Revert "SAUCE: i915_bpo: drm/i915/skl: DDI-E and DDI-A shares 4 lanes."
    - LP: #1490038

linux (3.19.0-27.29) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1485113

  [ Adam Lee ]

  * SAUCE: serial: 8250_pci: Add support for Pericom PI7C9X795[1248]
    - LP: #1480142

  [ Arun Siluvery ]

  * SAUCE: i915_bpo: drm/i915/gen8: Add infrastructure to initialize WA
    batch buffers
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen8: Re-order init pipe_control in lrc mode
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen8: Add WaDisableCtxRestoreArbitration
    workaround
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen8: Add
    WaFlushCoherentL3CacheLinesAtContextSwitch workaround
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915: Bail out early if WA batch is not available
    for given Gen
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen8: Add WaClearSlmSpaceAtContextSwitch
    workaround
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915: Update
    WaFlushCoherentL3CacheLinesAtContextSwitch
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915: Enable WA batch buffers for Gen9
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen9: Add WaDisableCtxRestoreArbitration
    workaround
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915: Update wa_ctx_emit() macro as per kernel
    coding guidelines
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen9: Add
    WaFlushCoherentL3CacheLinesAtContextSwitch workaround
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915/gen9: Add
    WaSetDisablePixMaskCammingAndRhwoInCommonSliceChicken
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915:skl: Add WaEnableGapsTsvCreditFix
    - LP: #1484486

  [ Chris Wilson ]

  * SAUCE: i915_bpo: drm/i915: Use two 32bit reads for select 64bit
    REG_READ ioctls
    - LP: #1484482
  * SAUCE: i915_bpo: drm/i915: Replace WARN inside I915_READ64_2x32 with
    retry loop
    - LP: #1484482
  * SAUCE: i915_bpo: drm/i915: Mark PIN_USER binding as GLOBAL_BIND without
    the aliasing ppgtt
    - LP: #1484482
  * SAUCE: i915_bpo: drm/i915: Declare the swizzling unknown for L-shaped
    configurations
    - LP: #1484482

  [ Damien Lespiau ]

  * SAUCE: i915_bpo: drm/i915/skl: Don't expose the top most plane on gen9
    display
    - LP: #1484486

  [ Daniel Vetter ]

  * SAUCE: i915_bpo: drm/i915: Fixup dp mst encoder selection
    - LP: #1484482

  [ David Weinehall ]

  * SAUCE: i915_bpo: drm/i915: Allow parsing of variable size child device
    entries from VBT
  * SAUCE: i915_bpo: drm/i915: Allow parsing of variable size child device
    entries from VBT, addendum v2
    - LP: #1484482

  [ Jani Nikula ]

  * SAUCE: i915_bpo: drm/i915/skl: WaIgnoreDDIAStrap is forever, always
    init DDI A
    - LP: #1484486
  * SAUCE: i915_bpo: drm/i915: reduce indent in i9xx_hpd_irq_handler
    - LP: #1484531
  * SAUCE: i915_bpo: drm/i91...

[oleg (overlayfs)]

Bug #1507463 has been filed for the 3.13 kernel in Ubuntu Trusty, which is also affected and has not yet been patched.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.