CONFIG_KVM is disabled for linux-raspi2 (aarch64)

[Colin Finck]

In contrast to the Linux kernel for x86_64, the CONFIG_KVM option is disabled for the "linux-raspi2" kernel (version 4.15.0-1016-raspi2 aarch64) on Ubuntu 18.04.
This prevents running QEMU with the -enable-kvm option to use hardware virtualization capabilities of the CPU.

I have recompiled the kernel with CONFIG_KVM set and could successfully run QEMU with -enable-kvm on my Raspberry Pi 3 B+ afterwards.
In my opinion, there is no reason for not activating CONFIG_KVM in the official raspi2 kernel.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Oliver Grawert (ogra)]

enabling CONFIG_KVM instead of enabling KVM via the dtb overlay will hard-disable all video acceleration of the upstream proprietary driver of the pi and make it completely unusable for any kind of setup that uses any upstram acceleration features from the raspberry pi foundation (i.e. via /opt/vc4 libs).

the proper way to enable KVM support on any pi is to set:

dtoverlay=vc4-kms-v3d

in the config.txt file of the bootloader, the kernel is specifically set up for this via the included raspberry pi foundation patches.

this might be pointless on arm64 which is not supported at all upstream by the raspberry pi foundation (beyond offering completely unsupported 64bit binaries for for developer tinkering) and thus does not offer any video acceleration via the proprietary driver anyway on this architecture.

but enabling CONFIG_KVM hardcoded in the kernel config for armhf will be fatal, please make sure this does not happen so users can still use vc4 accelerated video playback by manipulating the upstream-documented dtoverlay values on armhf builds.

[justmaybe (justmaybe)]

ogra: Hi, this doesn't work for ARM64... only enabling CONFIG_KVM does.

```ubuntu@ubuntu:/$ cat /boot/firmware/config.txt
#enable_uart=1
dtoverlay=vc4-kms-v3d
kernel=kernel8.bin
device_tree_address=0x03000000
dtparam=i2c_arm=on
dtparam=spi=on
dtparam=audio=on
arm_64bit=1
arm_freq=1100
arm_freq_min=400
gpu_mem=32
#gpu_freq=250
avoid_warnings=2
ubuntu@ubuntu:/$ ls /dev/k*
/dev/kmsg
ubuntu@ubuntu:/$```

Thanks,

[Oliver Grawert (ogra)]

just as a followup ... better ignore my comment above ... KVM is indeed not KMS ... sorry for causing confusion here

[Rohit Yadav (rohityadav)]

Just found this bug, I had added a similar bug ticket: https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1849369 (now marked as duplicate).

Copy paste from above:

Using the official Ubuntu 19.10 arm64 image, http://cdimage.ubuntu.com/ubuntu/releases/19.10/release/ubuntu-19.10-preinstalled-server-arm64+raspi3.img.xz, with the 5.3.x Linux kernel has KVM/VIRTUALIZATION disabled by default and therefore /dev/kvm acceleration is not available on installations on newer Raspberry Pi 4 (for example) which has a supported processor (armv8) by KVM (https://www.linux-kvm.org/page/Processor_support#ARM:).

I'm able to compile the kernel manually with the options enabled in the kernel config (http://dl.rohityadav.cloud/cloudstack-rpi/kernel-19.10/) and further verify that it works well in deploying ARM64 VMs (http://dl.rohityadav.cloud/cloudstack-rpi/template/). I went a bit further to deploy an Apache CloudStack based IaaS deployment and verify that it works as well. Having KVM enabled arm64 devices would enable testing and misc use-cases on newer boards such as the Raspberry Pi 4 (4 GB RAM).

Can the Ubuntu kernel team advise why KVM is not enabled in the kernel by default, and if this will change in future? Thanks.

[Hui Wang (hui.wang)]

I built a testing kernel which enabled the KVM, please test it on rpi4/3 (arm64) and rpi4/3/2 (armhf) boards.

https://people.canonical.com/~hwang4/pi4kvm/

thx.

[Rohit Yadav (rohityadav)]

Great, thanks @hui.wang. I'll test it in the evening and keep you posted.

[Rohit Yadav (rohityadav)]

@hui.wang - I tested the compiled kernel on my RaspberryPi4 (4GB RAM model). Both KVM and external SSD/USB (in my case Samsung T5, 500GB external SSD) on the USB 3.0 port are working fine.

root@cloudstack-mgmt:~# ls /dev/kvm
/dev/kvm
root@cloudstack-mgmt:~# kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Further description of how KVM was tested is here along with Apache CloudStack (an IaaS platform):
https://github.com/rhtyd/rhtyd.github.io/blob/master/_drafts/2019-10-22-cloudstack-rpi4-kvm.markdown

Tested using guest templates Debian 9 and Debian 10, both arm64 - http://dl.rohityadav.cloud/cloudstack-rpi/template/

Tested/issues:
- Only scsi seems to be the working disk controller (ide is not supported, virtio may/may not work) - when testing with linux guests. Workaround: use scsi by default.
- Plugging in usb devices without an explicit controller added first, fails (on x86_64 systems, controller seems to be already added when usb devices like keyboard/mouse etc are added). Workaround: add a usb controller and then add the usb devices - keyboard, mouse, tablet.

I've attached some screenshots of a guest VM running on the RPi4 device using CloudStack.

[Rohit Yadav (rohityadav)]

@hui.wang and the Ubuntu kernel team - kindly consider enabling KVM for arm64 Linux kernel in the raspberrypi preinstalled image in future. Thanks!

[Hui Wang (hui.wang)]

Will enable the KVM in the kernel of the next release.

[Hui Wang (hui.wang)]

For armhf (arm32) kernel, only CORTEX-A7 and CORTEXT-A15 support KVM, for Pi's cpu, they don't support KVM for armhf kernel.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-raspi2 - 5.3.0-1012.14

---------------
linux-raspi2 (5.3.0-1012.14) eoan; urgency=medium

  [ Ubuntu: 5.3.0-22.24 ]

  * [REGRESSION] md/raid0: cannot assemble multi-zone RAID0 with default_layout
    setting (LP: #1849682)
    - Revert "md/raid0: avoid RAID0 data corruption due to layout confusion."
  * refcount underflow and type confusion in shiftfs (LP: #1850867) // CVE-2019-15793
    - SAUCE: shiftfs: Correct id translation for lower fs operations
    - SAUCE: shiftfs: prevent type confusion
    - SAUCE: shiftfs: Fix refcount underflow in btrfs ioctl handling
  * CVE-2018-12207
    - kvm: x86, powerpc: do not allow clearing largepages debugfs entry
    - SAUCE: KVM: vmx, svm: always run with EFER.NXE=1 when shadow paging is
      active
    - SAUCE: x86: Add ITLB_MULTIHIT bug infrastructure
    - SAUCE: kvm: mmu: ITLB_MULTIHIT mitigation
    - SAUCE: kvm: Add helper function for creating VM worker threads
    - SAUCE: kvm: x86: mmu: Recovery of shattered NX large pages
    - SAUCE: cpu/speculation: Uninline and export CPU mitigations helpers
    - SAUCE: kvm: x86: mmu: Apply global mitigations knob to ITLB_MULTIHIT
  * CVE-2019-11135
    - x86/msr: Add the IA32_TSX_CTRL MSR
    - x86/cpu: Add a helper function x86_read_arch_cap_msr()
    - x86/cpu: Add a "tsx=" cmdline option with TSX disabled by default
    - x86/speculation/taa: Add mitigation for TSX Async Abort
    - x86/speculation/taa: Add sysfs reporting for TSX Async Abort
    - kvm/x86: Export MDS_NO=0 to guests when TSX is enabled
    - x86/tsx: Add "auto" option to the tsx= cmdline parameter
    - x86/speculation/taa: Add documentation for TSX Async Abort
    - x86/tsx: Add config options to set tsx=on|off|auto
    - [Config] Disable TSX by default when possible
  * CVE-2019-0154
    - SAUCE: drm/i915: Lower RM timeout to avoid DSI hard hangs
    - SAUCE: drm/i915/gen8+: Add RC6 CTX corruption WA
  * CVE-2019-0155
    - SAUCE: drm/i915: Rename gen7 cmdparser tables
    - SAUCE: drm/i915: Disable Secure Batches for gen6+
    - SAUCE: drm/i915: Remove Master tables from cmdparser
    - SAUCE: drm/i915: Add support for mandatory cmdparsing
    - SAUCE: drm/i915: Support ro ppgtt mapped cmdparser shadow buffers
    - SAUCE: drm/i915: Allow parsing of unsized batches
    - SAUCE: drm/i915: Add gen9 BCS cmdparsing
    - SAUCE: drm/i915/cmdparser: Use explicit goto for error paths
    - SAUCE: drm/i915/cmdparser: Add support for backward jumps
    - SAUCE: drm/i915/cmdparser: Ignore Length operands during command matching

 -- Juerg Haefliger <email address hidden> Mon, 11 Nov 2019 08:19:54 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-raspi2 - 5.3.0-1014.16

---------------
linux-raspi2 (5.3.0-1014.16) eoan; urgency=medium

  * eoan/linux-raspi2: 5.3.0-1014.16 -proposed tracker (LP: #1854006)

  * Need to disable CONFIG_DRＭ_V3D in the raspi2 eoan kernel (LP: #1853789)
    - [config] raspi2: Revert "UBUNTU: [config] raspi2: CONFIG_DRM_V3D=m"

linux-raspi2 (5.3.0-1013.15) eoan; urgency=medium

  * eoan/linux-raspi2: 5.3.0-1013.15 -proposed tracker (LP: #1852220)

  * Eoan update: 5.3.9 upstream stable release (LP: #1851550)
    - raspi2: [Config] Remove CONFIG_GENERIC_COMPAT_VDSO and
      CONFIG_CROSS_COMPILE_COMPAT_VDSO

  * Eoan update: v5.3.8 upstream stable release (LP: #1850456)
    - raspi2: [Config] CAVIUM_TX2_ERRATUM_219=n

  * IO errors when writing large amounts of data to USB storage in eoan on
    RPI2/3 (armhf kernel) (LP: #1852510)
    - SAUCE: dwc_otg: checking the urb->transfer_buffer too early (#3332)

  * Incorrect raspi2 snapcraft.yaml file (LP: #1851469)
    - [Packaging] raspi2: Fix snapcraft.yaml

  * CONFIG_DRＭ_V3D is disabled for linux-raspi2 of eoan (LP: #1850876)
    - [config] raspi2: CONFIG_DRM_V3D=m

  [ Ubuntu: 5.3.0-24.26 ]

  * eoan/linux: 5.3.0-24.26 -proposed tracker (LP: #1852232)
  * Eoan update: 5.3.9 upstream stable release (LP: #1851550)
    - io_uring: fix up O_NONBLOCK handling for sockets
    - dm snapshot: introduce account_start_copy() and account_end_copy()
    - dm snapshot: rework COW throttling to fix deadlock
    - Btrfs: fix inode cache block reserve leak on failure to allocate data space
    - btrfs: qgroup: Always free PREALLOC META reserve in
      btrfs_delalloc_release_extents()
    - iio: adc: meson_saradc: Fix memory allocation order
    - iio: fix center temperature of bmc150-accel-core
    - libsubcmd: Make _FORTIFY_SOURCE defines dependent on the feature
    - perf tests: Avoid raising SEGV using an obvious NULL dereference
    - perf map: Fix overlapped map handling
    - perf script brstackinsn: Fix recovery from LBR/binary mismatch
    - perf jevents: Fix period for Intel fixed counters
    - perf tools: Propagate get_cpuid() error
    - perf annotate: Propagate perf_env__arch() error
    - perf annotate: Fix the signedness of failure returns
    - perf annotate: Propagate the symbol__annotate() error return
    - perf annotate: Fix arch specific ->init() failure errors
    - perf annotate: Return appropriate error code for allocation failures
    - perf annotate: Don't return -1 for error when doing BPF disassembly
    - staging: rtl8188eu: fix null dereference when kzalloc fails
    - RDMA/siw: Fix serialization issue in write_space()
    - RDMA/hfi1: Prevent memory leak in sdma_init
    - RDMA/iw_cxgb4: fix SRQ access from dump_qp()
    - RDMA/iwcm: Fix a lock inversion issue
    - HID: hyperv: Use in-place iterator API in the channel callback
    - kselftest: exclude failed TARGETS from runlist
    - selftests/kselftest/runner.sh: Add 45 second timeout per test
    - nfs: Fix nfsi->nrequests count error on nfs_inode_remove_request
    - arm64: cpufeature: Effectively expose FRINT capability to userspace
    - arm64: Fix incorrect irqflag restore for priority masking fo...