Sluggish performance as Xen dom0

[小太]

1) The release of Ubuntu you are using, via 'lsb_release -rd' or System -> About Ubuntu

16.04 LTS

2) The version of the package you are using, via 'apt-cache policy pkgname' or by checking in Software Center

Tried booting various versions between the 4.13.0-lowlatency and 4.15.0-lowlatency versions.
Can confirm the problem appears on all 4.15.0 versions (e.g., 4.15.0-13), but is not present on any 4.13.0 version (e.g., 4.13.0-45).

The version of Xen used is 4.9.0-0ubuntu3. I believe this isn't in Xenial's repositories, but actually from a later Ubuntu release, since I wanted some features of Xen that weren't in Xenial's version.

Bisecting the commits narrowed down to the issue appearing after http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/commit/?h=hwe-edge&id=a62a7117d91ca83d319566cbe16039f4e9f413c2 was committed.

In fact, after recompiling and running the latest hwe-edge kernel (4.15.0-23) with that commit reverted, the problem disappears. This kernel is attached.

I also intend to test the mainline kernel, but I haven't gotten around to it yet

3/4) What you expected to happen/What happened instead
The newer kernel to not be significantly slower than the older kernel (I'm aware of Meltdown/Spectre mitigations causing slowdowns, but it should not be *this* much slower).

Specifically, after upgrading to a 4.15.0 kernel from 4.13.0, I noticed these behaviours:
- Time to log in via lightdm seems to have doubled in time or more (I did not actually time it, but it is taking significantly longer)
- Scrolling in the Processes tab of system-monitor is noticeably choppy and laggy
- The "user experience" in general seems to be laggy
- Running a game in a Windows domU (with passthrough GPU) resulted in ~25% of the original FPS

Interestingly, if I boot on bare metal (i.e., without Xen), the problem does not manifest.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.15.0-22-lowlatency 4.15.0-22.24~16.04.1
ProcVersionSignature: Ubuntu 4.15.0-23.25~16.04.1-lowlatency 4.15.18
Uname: Linux 4.15.0-23-lowlatency x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
CurrentDesktop: GNOME-Flashback:Unity
Date: Fri Jun 29 21:46:35 2018
SourcePackage: linux-signed-hwe-edge
UpgradeStatus: No upgrade log present (probably fresh install)

[小太 (nospam-i)]

Note that this dmesg is taken under my custom 4.15.0-23-lowlatency kernel with the offending patch reverted.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[小太 (nospam-i)]

I tested the upstream v4.18-rc2 kernel, and found the problem has been fixed.

Bisecting revealed https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5a8361f7ecceaed64b4064000d16cb703462be49 fixed the issue, which is included in v4.16.0.

Given that the most recent Ubuntu kernel seems to be v4.15.y, when can I expect that commit to be included in an Ubuntu kernel release? Would the soonest likely be when 18.10 is released?

[小太 (nospam-i)]

Correction: The above commit was included in v4.17, not v4.16.

[小太 (nospam-i)]

For future reference purposes, here are some dmesg logs between v4.16.18 (bug not fixed) and v4.17.3 (bug fixed)