Unable to build lttng-module with Artful kernel

Bug #1756240 reported by Po-Hsu Lin
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Invalid
Undecided
Khaled El Mously
Xenial
Invalid
Undecided
Khaled El Mously
Artful
Invalid
Undecided
Khaled El Mously
Bionic
Invalid
Undecided
Khaled El Mously
lttng-modules (Ubuntu)
Fix Released
Undecided
Khaled El Mously
Xenial
Fix Released
Undecided
Khaled El Mously
Artful
Fix Released
Undecided
Khaled El Mously
Bionic
Fix Released
Undecided
Khaled El Mously

Bug Description

[Impact]

 * The 4.13.0-38 artful kernel included a change (backported from 4.15) that modifies the definition of kvm_mmio(). This breaks the lttng-modules-dkms package because it is using the old (<4.15) definition.

 * lttng-modules-dkms already has a list of kernel version "exceptions" with which it uses the >=4.15 definition. Proposed solution is to add the Ubuntu kernel range 4.13.0-38 up to 4.14.255-255 to that list of exceptions.

[Test Case]

 * Tested by first successfully building lttng-modules-dkms on 4.13.0-37, then reproducing the problem described here on 4.13.0-38, then applying the code-change and confirming that I can successfully build on both 4.13.0-37 and 4.13.0-38 after that.

[Regression Potential]

 * Low-regression potential. Could prevent lttng-modules-dkms from being built for specific kernel versions, but this is unlikely as the change was tested to confirm that it works as expected on 4.13.0-37 and 4.13.0-38 - and should work fine for other Ubuntu kernels as well.

The lttng module failed to build with the proposed Artful kernel (4.13.0-38), it works with 4.13.0-37

Please find the build log here: http://paste.ubuntu.com/p/ybwnBJB9yj/

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: linux-image-4.13.0-38-generic 4.13.0-38.43
ProcVersionSignature: User Name 4.13.0-38.43-generic 4.13.16
Uname: Linux 4.13.0-38-generic x86_64
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Mar 16 04:13 seq
 crw-rw---- 1 root audio 116, 33 Mar 16 04:13 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.7-0ubuntu3.7
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CurrentDmesg:

Date: Fri Mar 16 04:17:14 2018
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
MachineType: Intel Corporation S1200RP
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB:
 0 mgadrmfb
 1 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.13.0-38-generic root=UUID=b0d2ae4e-12dd-423e-acea-272ee8b2a893 ro
RelatedPackageVersions:
 linux-restricted-modules-4.13.0-38-generic N/A
 linux-backports-modules-4.13.0-38-generic N/A
 linux-firmware 1.169.3
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 07/01/2015
dmi.bios.vendor: Intel Corp.
dmi.bios.version: S1200RP.86B.03.02.0003.070120151022
dmi.board.asset.tag: ....................
dmi.board.name: S1200RP
dmi.board.vendor: Intel Corporation
dmi.board.version: G62254-407
dmi.chassis.asset.tag: ....................
dmi.chassis.type: 17
dmi.chassis.vendor: ..............................
dmi.chassis.version: ..................
dmi.modalias: dmi:bvnIntelCorp.:bvrS1200RP.86B.03.02.0003.070120151022:bd07/01/2015:svnIntelCorporation:pnS1200RP:pvr....................:rvnIntelCorporation:rnS1200RP:rvrG62254-407:cvn..............................:ct17:cvr..................:
dmi.product.family: To be filled by O.E.M.
dmi.product.name: S1200RP
dmi.product.version: ....................
dmi.sys.vendor: Intel Corporation

CVE References

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :
Changed in linux (Ubuntu):
assignee: nobody → Colin Ian King (colin-king)
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote : Missing required logs.

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1756240

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

This issue can be reproduced with Xenial HWE kernel as well (4.13)

03/20 05:22:06 DEBUG| utils:0153| [stdout] Building initial module for 4.15.0-13-lowlatency
03/20 05:22:14 DEBUG| utils:0153| [stdout] Error! Bad return status for module build on kernel: 4.15.0-13-lowlatency (x86_64)
03/20 05:22:14 DEBUG| utils:0153| [stdout] Consult /var/lib/dkms/lttng-modules/2.8.0/build/make.log for more information.

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Sorry in comment #3 it's the Xenial Edge kernel (4.15), this is the Xenial HWE one:

03/16 01:12:07 DEBUG| utils:0153| [stdout] Error! Bad return status for module build on kernel: 4.13.0-38-lowlatency (x86_64)
03/16 01:12:07 DEBUG| utils:0153| [stdout] Consult /var/lib/dkms/lttng-modules/2.8.0/build/make.log for more information.
03/16 01:12:07 DEBUG| utils:0153| [stdout] Setting up lttng-tools (2.7.1-2ubuntu1) ...

Changed in linux (Ubuntu):
status: Incomplete → In Progress
Revision history for this message
Colin Ian King (colin-king) wrote :

This build failure occurs because of the following artful commit:

commit 356852284931cda6749c41f77acefcb13bd534fe
Author: Wanpeng Li <email address hidden>
Date: Thu Jan 4 01:57:52 2018 -0500

    KVM: Fix stack-out-of-bounds read in write_mmio

    CVE-2017-17741

This redefines the kvm_mmio trace event as follows:

 TRACE_EVENT(kvm_mmio,
- TP_PROTO(int type, int len, u64 gpa, u64 val),
+ TP_PROTO(int type, int len, u64 gpa, void *val),
        TP_ARGS(type, len, gpa, val),

..which is different from the lttng prototype in instrumentation/events/lttng-module/kvm.h:

#if (LINUX_VERSION_CODE >= KERNEL_VERSION(4,15,0) \
        || LTTNG_KERNEL_RANGE(4,14,14, 4,15,0) \
        || LTTNG_KERNEL_RANGE(4,9,77, 4,10,0) \
        || LTTNG_KERNEL_RANGE(4,4,112, 4,5,0) \
        || LTTNG_KERNEL_RANGE(3,16,52, 3,17,0) \
        || LTTNG_KERNEL_RANGE(3,2,97, 3,3,0) \
        || LTTNG_DEBIAN_KERNEL_RANGE(4,9,65,0,3,0, 4,10,0,0,0,0))

LTTNG_TRACEPOINT_EVENT(kvm_mmio,
        TP_PROTO(int type, int len, u64 gpa, void *val),
        TP_ARGS(type, len, gpa, val),

        TP_FIELDS(
                ctf_integer(u32, type, type)
                ctf_integer(u32, len, len)
                ctf_integer(u64, gpa, gpa)
                ctf_sequence_hex(unsigned char, val, val, u32, len)
        )
)

#else

LTTNG_TRACEPOINT_EVENT(kvm_mmio,
        TP_PROTO(int type, int len, u64 gpa, u64 val),
        TP_ARGS(type, len, gpa, val),

        TP_FIELDS(
                ctf_integer(u32, type, type)
                ctf_integer(u32, len, len)
                ctf_integer(u64, gpa, gpa)
                ctf_integer(u64, val, val)
        )
)

Not sure how we #ifdef around this as the kernel version is 4.13 yet it now has a 4.15 API

Brad Figg (brad-figg)
Changed in linux (Ubuntu):
assignee: Colin Ian King (colin-king) → nobody
Revision history for this message
Khaled El Mously (kmously) wrote :

Can we fix this by modifying the code in lttng-modules-dkms ? Something like "proposed-patch-1.txt"

Changed in linux (Ubuntu):
assignee: nobody → Khaled El Mously (kmously)
tags: added: patch
Revision history for this message
Khaled El Mously (kmously) wrote :

4.13.0-38 introduced a new definition of kvm_mmio() that is backported from 4.15. There's already a list of excepted kernel versions for which the new (4.15) definition should be used. I added ubuntu 4.13.0-38-and-later kernels to that list.

Tested by first successfully building lttng-modules-dkms on 4.13.0-37, then reproducing the problem described here on 4.13.0-38, then applying the code-change and confirming that I can successfully build on both 4.13.0-37 and 4.13.0-38 after that.

description: updated
Revision history for this message
Khaled El Mously (kmously) wrote :

Updated debdiff to fix a problem with macro arguments. Same testing as before.

Revision history for this message
Khaled El Mously (kmously) wrote :

Adding equivalent fix for xenial-hwe

Revision history for this message
Tyler Hicks (tyhicks) wrote :

@Khaled these debdiffs look good enough for me to sponsor. I made one superficial change by removing the "## " prefix from the patch tags lines. However, there were two important changes required for the Xenial debdiff. The version wasn't correct (I changed it to 2.8.0-1ubuntu1~16.04.5 instead of 2.8.0-1ubuntu1~16.04.4ubuntu1) and the changelog wasn't finalized (I changed "UNRELEASED" to "xenial"). Thanks for preparing the debdiffs and please remember these two things next time!

@sru-team I asked Khaled, over IRC, if these changes were need in Bionic. He says, "I don't think so - bionic would take the 4.15 definition by default. So the problem would only affect ubuntu kernels version 4.13.0-38 up to (but not including) 4.15.x-y".

I've sponsored these (adjusted) debdiffs to artful and xenial. Thanks again!

Revision history for this message
Tyler Hicks (tyhicks) wrote :

@Khaled after giving it a little more thought, I think these changes might be needed in bionic in the situation that an artful user is upgrading to bionic or when a xenial hwe user is upgrading to bionic. I think that if the lttng-modules-dkms binary package is upgraded to the bionic package bionic while there are still kernels installed from the old release, the user will see this bug. Could you test these situations or just get out a bigger hammer and prepare a bionic debdiff that I can sponsor?

Changed in linux (Ubuntu Artful):
assignee: nobody → Khaled El Mously (kmously)
Changed in linux (Ubuntu Xenial):
assignee: nobody → Khaled El Mously (kmously)
status: New → In Progress
Changed in linux (Ubuntu Artful):
status: New → In Progress
Changed in linux (Ubuntu Bionic):
status: In Progress → New
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1756240

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Tyler Hicks (tyhicks)
Changed in lttng-modules (Ubuntu Xenial):
status: New → In Progress
Changed in lttng-modules (Ubuntu Artful):
status: New → In Progress
Changed in lttng-modules (Ubuntu Xenial):
assignee: nobody → Khaled El Mously (kmously)
Changed in lttng-modules (Ubuntu Artful):
assignee: nobody → Khaled El Mously (kmously)
Changed in lttng-modules (Ubuntu Bionic):
assignee: nobody → Khaled El Mously (kmously)
Changed in linux (Ubuntu Xenial):
status: In Progress → Invalid
Changed in linux (Ubuntu Artful):
status: In Progress → Invalid
Changed in linux (Ubuntu Bionic):
status: Incomplete → Invalid
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "proposed-fix" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

Revision history for this message
Khaled El Mously (kmously) wrote :

Fix for bionic - lots of thanks to @tyhicks for the explanation and help

Revision history for this message
Tyler Hicks (tyhicks) wrote :

lgtm! sponsored to bionic

Changed in lttng-modules (Ubuntu Bionic):
status: New → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lttng-modules - 2.10.5-1ubuntu1

---------------
lttng-modules (2.10.5-1ubuntu1) bionic; urgency=medium

  * Fix builds for 4.13.0-38 kernel (LP: #1756240)
    - Update definition of kvm_mmio() to match the new kernel

 -- Khalid Elmously <email address hidden> Fri, 23 Mar 2018 10:20:08 -0400

Changed in lttng-modules (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Po-Hsu, or anyone else affected,

Accepted lttng-modules into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lttng-modules/2.9.0-1ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in lttng-modules (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Revision history for this message
Khaled El Mously (kmously) wrote :

Fix has been merged into mainline lttng as well:

git://git.lttng.org/lttng-modules.git be02509026192106196fe6dfd9df0db739804045

Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

The fix works as expected, thanks.

tags: added: verification-done verification-done-artful
removed: verification-needed verification-needed-artful
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Po-Hsu, or anyone else affected,

Accepted lttng-modules into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/lttng-modules/2.8.0-1ubuntu1~16.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in lttng-modules (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
removed: verification-done
Revision history for this message
Po-Hsu Lin (cypressyew) wrote :

Tested on Xenial + 4.13 kernel, works as expected.
Thanks

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Khaled El Mously (kmously) wrote :

Hello SRU team,

Verification appears to be complete for this fix and the holding period has ended. Could you please promote this to -updates?
Thanks

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lttng-modules - 2.9.0-1ubuntu3.2

---------------
lttng-modules (2.9.0-1ubuntu3.2) artful; urgency=medium

  * Fix builds for 4.13.0-38 kernel (LP: #1756240)
    - Update definition of kvm_mmio() to match the new kernel

 -- Khalid Elmously <email address hidden> Wed, 21 Mar 2018 19:12:27 -0400

Changed in lttng-modules (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for lttng-modules has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lttng-modules - 2.8.0-1ubuntu1~16.04.5

---------------
lttng-modules (2.8.0-1ubuntu1~16.04.5) xenial; urgency=medium

  * Fix builds for 4.13.0-38 kernel (LP: #1756240)
    - Update definition of kvm_mmio() to match the new kernel

 -- Khalid Elmously <email address hidden> Thu, 22 Mar 2018 12:08:10 -0400

Changed in lttng-modules (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.