lxcfs: update the 'btime' field in /proc/stat to reflect guest boot time not host

Bug #1654310 reported by Jason Baron on 2017-01-05
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
lxcfs (Ubuntu)
Medium
Unassigned
Xenial
High
Unassigned
Yakkety
Undecided
Unassigned
Zesty
Undecided
Unassigned

Bug Description

The 'btime' field in /proc/stat reflects the host boot time and not the time that the guest container has started.

affects: make-dfsg (Ubuntu) → lxcfs (Ubuntu)
Serge Hallyn (serge-hallyn) wrote :

Thanks I've created https://github.com/lxc/lxcfs/issues/164 to track this.

Changed in lxcfs (Ubuntu):
importance: Undecided → Medium
status: New → Triaged
Jason Baron (jibaron) wrote :

I've created a pull request to address this issue:
https://github.com/lxc/lxcfs/pull/169

Thanks.

Evgeni Golov (evgeni) wrote :

This is fixed in LXCFS 2.0.7

Changed in lxcfs (Ubuntu):
status: Triaged → Fix Released
gozdal (gozdal) wrote :

This introduces a different problem: process creation time is counted as offset from btime:

(22) starttime %llu
          The time the process started after system boot. In
          kernels before Linux 2.6, this value was expressed
          in jiffies. Since Linux 2.6, the value is expressed
          in clock ticks (divide by sysconf(_SC_CLK_TCK)).

(http://man7.org/linux/man-pages/man5/proc.5.html)

This means that because starttime is not virtualized, all the processes within the LXC container appear as time travellers from the future.

On Mon, Jun 19, 2017 at 01:04:38PM -0000, gozdal wrote:
> This introduces a different problem: process creation time is counted as
> offset from btime:
>
> (22) starttime %llu
> The time the process started after system boot. In
> kernels before Linux 2.6, this value was expressed
> in jiffies. Since Linux 2.6, the value is expressed
> in clock ticks (divide by sysconf(_SC_CLK_TCK)).

Yes, we reverted that patch since we consider STTIME to be more important.

>
> (http://man7.org/linux/man-pages/man5/proc.5.html)
>
> This means that because starttime is not virtualized, all the processes
> within the LXC container appear as time travellers from the future.
>
> --
> You received this bug notification because you are a member of Ubuntu
> containers team, which is subscribed to lxcfs in Ubuntu.
> Matching subscriptions: lxcfs
> https://bugs.launchpad.net/bugs/1654310
>
> Title:
> lxcfs: update the 'btime' field in /proc/stat to reflect guest boot
> time not host
>
> Status in lxcfs package in Ubuntu:
> Fix Released
>
> Bug description:
> The 'btime' field in /proc/stat reflects the host boot time and not
> the time that the guest container has started.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/lxcfs/+bug/1654310/+subscriptions

Jason Baron (jibaron) wrote :

Ok, I guess we could intercept the /proc/<pid>/stat read() call and overwrite the starttime there with the correct value? That is, add the starttime from /proc/<pid>/stat to the host boot time and then subtract that from the guest boot time.

Christian Brauner (cbrauner) wrote :

On Mon, Jun 19, 2017 at 04:28:07PM -0000, Jason Baron wrote:
> Ok, I guess we could intercept the /proc/<pid>/stat read() call and
> overwrite the starttime there with the correct value? That is, add the
> starttime from /proc/<pid>/stat to the host boot time and then subtract
> that from the guest boot time.

We can't really do that since this is equivalent to over-mounting /proc itself.
Instead, we have reverted the "btime" virtualization patch for now and we'll
likely include this in the next round of SRUs.

>
> --
> You received this bug notification because you are a member of Ubuntu
> containers team, which is subscribed to lxcfs in Ubuntu.
> Matching subscriptions: lxcfs
> https://bugs.launchpad.net/bugs/1654310
>
> Title:
> lxcfs: update the 'btime' field in /proc/stat to reflect guest boot
> time not host
>
> Status in lxcfs package in Ubuntu:
> Fix Released
>
> Bug description:
> The 'btime' field in /proc/stat reflects the host boot time and not
> the time that the guest container has started.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/lxcfs/+bug/1654310/+subscriptions

gozdal (gozdal) wrote :

I believe that packages for several distributions (esp. Xenial) still contain this now-revered patch. Would it be possible to release packages with btime virtualization reverted?

Serge Hallyn (serge-hallyn) wrote :

@gozdal,

would you be willing to add a SRU justification as per section 3.1 of https://wiki.ubuntu.com/StableReleaseUpdates to the description of this bug?

Changed in lxcfs (Ubuntu Xenial):
importance: Undecided → High
Serge Hallyn (serge-hallyn) wrote :

I've pushed a package reverting that field to xenial-proposed - it will await approval there.

I've set the priority to high as this seems to be affecting other software adversely.

gozdal (gozdal) wrote :

[Impact]

Processes within LXC container are seen as created in the future.

[Test Case]

1. create an LXC container on a host which has been up for untrivial amount of time (say 1 hour)
2. look at "ps aux" output in START column - all the process are seen as started in the future

[Regression Potential]

Software comparing process start time to any other time may fail in unexpected ways. As an example some software compare process start time to PID file timestamp on disk and disregard old PID files. In LXC this code will disregard all PID files.

Hello Jason, or anyone else affected,

Accepted lxcfs into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxcfs/2.0.7-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in lxcfs (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed verification-needed-xenial
Changed in lxcfs (Ubuntu Yakkety):
status: New → Fix Committed
tags: added: verification-needed-yakkety
Stéphane Graber (stgraber) wrote :

Hello Jason, or anyone else affected,

Accepted lxcfs into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxcfs/2.0.7-0ubuntu1~16.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-yakkety to verification-done-yakkety.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-yakkety. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in lxcfs (Ubuntu Zesty):
status: New → Fix Committed
tags: added: verification-needed-zesty
Stéphane Graber (stgraber) wrote :

Hello Jason, or anyone else affected,

Accepted lxcfs into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lxcfs/2.0.7-0ubuntu1~17.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Brian Candler (b-candler) wrote :

After upgrading to lxcfs/xenial-proposed (2.0.7-0ubuntu1~16.04.2), the problem is fixed for me

Inside a container:

root@apt-cacher:~# ps auxwww | grep apt
apt-cac+ 309 0.0 0.7 653476 7128 ? Ssl Jul08 1:02 /usr/sbin/apt-cacher-ng SocketPath=/run/apt-cacher-ng/socket -c /etc/apt-cacher-ng ForeGround=1
root 1922 0.0 0.1 14620 1016 ? S+ 08:04 0:00 grep --color=auto apt
root@apt-cacher:~# date
Tue Jul 11 08:04:09 UTC 2017

- start time of apt-cacher-ng is correct (in the past)
- start time of 'grep' process is correct (current time)

tags: added: verification-done-xenial
removed: verification-needed-xenial
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxcfs - 2.0.7-0ubuntu1~16.04.2

---------------
lxcfs (2.0.7-0ubuntu1~16.04.2) xenial-proposed; urgency=medium

  * Cherrypick patch reverting the virtualization of bstat field (LP: #1654310)

 -- Serge Hallyn <email address hidden> Tue, 04 Jul 2017 10:43:18 -0500

Changed in lxcfs (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for lxcfs has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxcfs - 2.0.7-0ubuntu1~16.10.2

---------------
lxcfs (2.0.7-0ubuntu1~16.10.2) yakkety-proposed; urgency=medium

  * Cherrypick patch reverting the virtualization of bstat field (LP: #1654310)

 -- Serge Hallyn <email address hidden> Tue, 04 Jul 2017 13:29:14 -0500

Changed in lxcfs (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxcfs - 2.0.7-0ubuntu1~17.04.2

---------------
lxcfs (2.0.7-0ubuntu1~17.04.2) zesty-proposed; urgency=medium

  * Cherrypick patch reverting the virtualization of bstat field (LP: #1654310)

 -- Serge Hallyn <email address hidden> Tue, 04 Jul 2017 13:29:40 -0500

Changed in lxcfs (Ubuntu Zesty):
status: Fix Committed → Fix Released
Changed in lxcfs (Ubuntu):
status: Fix Released → Triaged
Changed in lxcfs (Ubuntu Xenial):
status: Fix Released → Triaged
Changed in lxcfs (Ubuntu Yakkety):
status: Fix Released → Triaged
Changed in lxcfs (Ubuntu Zesty):
status: Fix Released → Triaged
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxcfs - 2.0.8-1ubuntu1

---------------
lxcfs (2.0.8-1ubuntu1) bionic; urgency=medium

  * Re-sync with Debian, remaining delta (for backports):
    - Include upstart jobs on older Ubuntu releases
    - Build-depend on dh-systemd with older debootstrap

 -- Stéphane Graber <email address hidden> Tue, 31 Oct 2017 18:40:42 -0400

Changed in lxcfs (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.