systemd sometimes misdetects Xen VMs (fixed up-stream)

[dana]

[Impact]

 * XEN virtualization missidentified as hyperv
 * This may cause units limited to a virtualisation not to be started; or redundant units for the other type of virtualisation to be started instead.
 * Previously xen & hyper-v were detected correctly separately, even when xen emulates/suggests that it is hyper-v.

[Test Case]

 * Execute systemd-detect-virt and check that it detects Xen & Hyperv

[Regression Potential]

 * Things that "fake" xen might not work as the detection code branches are changed, but it would affect previous releases and future releases of systemd-detect-virt. So far such things are not known to exist, and if such things appear in the future support for them would need to land upstream first.

[Other Info]

 * Original bug report

Cherrypick https://github.com/systemd/systemd/commit/f2fe2865cd19cd4318b82d01b9b62d22b1697b3a

It appears that Xen-based VMs sometimes report themselves as Microsoft Hyper-V via CPUID — apparently this is for compatibility with Windows guests. systemd 237 (as found in Bionic) gives preference to this CPUID information when detecting Xen, and thus it erroneously assumes that the guest is running under Hyper-V. This causes Xen-related services (and anything else that relies on systemd's VM-detection functionality) to fail.

I *believe* this is a regression from systemd 229 as used in Xenial — we have at least a few Xen-based VMs that report as Hyper-V via CPUID and don't have this issue on that version — but i haven't confirmed that for certain.

Anyway, i've submitted a ticket with more details to the up-stream project, and that has now been resolved through a fairly simple change which applies cleanly to the Bionic systemd sources:

https://github.com/systemd/systemd/issues/8844

Would it be possible to pull this down? As mentioned, Xen-based Ubuntu VMs can seriously misbehave without it.

Thanks!

See also: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1728573

---

% lsb_release -rd
Description: Ubuntu 18.04 LTS
Release: 18.04

% apt-cache policy systemd
systemd:
  Installed: 237-3ubuntu10
  Candidate: 237-3ubuntu10
  Version table:
 *** 237-3ubuntu10 500
        500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        100 /var/lib/dpkg/status

[Łukasz Zemczak (sil2100)]

Hello dana, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dana (okdana)]

Hi Łukasz and Dimitri,

Thanks for responding so quickly. I've tested this on my Bionic VM and it seems to be working:

% lsb_release -sd
Ubuntu 18.04 LTS

% uname -r
4.15.0-20-generic

% dpkg-query -Wf '${Package}: ${Version}\n' | rg systemd
libnss-systemd: 237-3ubuntu10.1
libpam-systemd: 237-3ubuntu10.1
libsystemd0: 237-3ubuntu10.1
python3-systemd: 234-1build1
systemd: 237-3ubuntu10.1
systemd-sysv: 237-3ubuntu10

% SYSTEMD_LOG_LEVEL=debug systemd-detect-virt
Failed to read $container of PID 1, ignoring: Permission denied
Found container virtualization none.
Virtualization Xen found in DMI (/sys/class/dmi/id/sys_vendor)
Virtualization XEN, found /sys/hypervisor/properties/features with value 00002705, XENFEAT_dom0 (indicating the 'hardware domain') is not set.
Found VM virtualization xen
xen

I also did a few spot checks of journalctl, systemctl, &c., and all that looks OK too.

Thanks!

[Steve Langasek (vorlon)]

Hello dana, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[dana (okdana)]

Hi Steve,

This build seems to work OK for me as well. Thanks!

% dpkg-query -Wf '${Package}: ${Version}\n' | rg systemd
libnss-systemd: 237-3ubuntu10.2
libpam-systemd: 237-3ubuntu10.2
libsystemd0: 237-3ubuntu10.2
python3-systemd: 234-1build1
systemd: 237-3ubuntu10.2
systemd-sysv: 237-3ubuntu10

% SYSTEMD_LOG_LEVEL=debug systemd-detect-virt
Failed to read $container of PID 1, ignoring: Permission denied
Found container virtualization none.
Virtualization Xen found in DMI (/sys/class/dmi/id/sys_vendor)
Virtualization XEN, found /sys/hypervisor/properties/features with value 00002705, XENFEAT_dom0 (indicating the 'hardware domain') is not set.
Found VM virtualization xen
xen

% systemctl is-active xe-daemon
active

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 237-3ubuntu10.2

---------------
systemd (237-3ubuntu10.2) bionic; urgency=medium

  * logind: backport v238/v239 fixes for handling DRM devices.
    These changes introduce all the fixes that correct handling of open fd's
    related to the DRM devices, as used by for example NVIDIA GPUs. This backport
    includes some refactoring, corrections, and comment updates. This to insure
    that correct history is preserved, code comments match reality, and to ease
    backporting logind fixes in the future SRUs. (LP: #1777099)
  * Disable dh_installinit generation of tmpfiles for the systemd package.
    Replace with a manual safe call to systemd-tmpfiles which will process any
    updates to the tmpfiles shipped by systemd package, taking into account any
    overrides shipped by other packages, sysadmin, or specified in the runtime
    directories. (LP: #1748147)

systemd (237-3ubuntu10.1) bionic; urgency=medium

  [ Dimitri John Ledkov ]
  * hwdb: Fix wlan/rfkill keycode on Dell systems. (LP: #1762385)
  * Cherrypick upstream fix for corrected detection of Virtualbox & Xen.
    (LP: #1768104)
  * Further improve captive portal workarounds.
    Retry any NXDOMAIN results with lower feature levels, instead of just those
    with 'secure' in the domain name. (LP: #1766969)

  [ Michael Biebl ]
  * Add dependencies of libsystemd-shared to Pre-Depends.
    This is necessary so systemctl is functional at all times during a
    dist-upgrade. (Closes: #897986) (LP: #1771791)

  [ Mario Limonciello ]
  * Fix hibernate disk offsets.
    Configure resume offset via sysfs, to enable resume from a swapfile.
    (LP: #1760106)

 -- Dimitri John Ledkov 🌈 <email address hidden> Fri, 22 Jun 2018 13:55:09 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for systemd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 239-7ubuntu4

---------------
systemd (239-7ubuntu4) cosmic; urgency=medium

  * Workaround broken meson copying symlinked data files, as dangling symlinks.

 -- Dimitri John Ledkov <email address hidden> Wed, 22 Aug 2018 14:11:35 +0100