Can't Load vfio-pci Module on Ubuntu 18.04 HWE Kernel

[Youhei Tooyama (VirtualTech Japan)]

Can't Load vfio-pci Module on Ubuntu 18.04 HWE Kernel

```
ubuntu@ubuntu-2:~$ modinfo vfio-pci
modinfo: ERROR: Module vfio-pci not found.

ubuntu@ubuntu-2:~$ uname -a
Linux DL360G9-2 5.4.0-52-generic #57~18.04.1-Ubuntu SMP Thu Oct 15 14:04:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
```

But Generic kernel is Working.

```
ubuntu@ubuntu-4:~$ modinfo vfio-pci
filename: /lib/modules/4.15.0-122-generic/kernel/drivers/vfio/pci/vfio-pci.ko
description: VFIO PCI - User Level meta-driver
author: Alex Williamson <email address hidden>
license: GPL v2
version: 0.2
srcversion: 1E92A800B94EAE7B392AD50
depends: vfio,irqbypass,vfio_virqfd
retpoline: Y
intree: Y
name: vfio_pci
vermagic: 4.15.0-122-generic SMP mod_unload
signat: PKCS#7
signer:
sig_key:
sig_hashalgo: md4
parm: ids:Initial PCI IDs to add to the vfio driver, format is "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and multiple comma separated entries can be specified (string)
parm: nointxmask:Disable support for PCI 2.3 style INTx masking. If this resolves problems for specific devices, report lspci -vvvxxx to <email address hidden> so the device can be fixed automatically via the broken_intx_masking flag. (bool)
parm: disable_vga:Disable VGA resource access through vfio-pci (bool)
parm: disable_idle_d3:Disable using the PCI D3 low power state for idle, unused devices (bool)
```

How to use vfio-pci module in HWE kernel?
---
ProblemType: Bug
AlsaVersion: Advanced Linux Sound Architecture Driver Version k5.4.0-52-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.16
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', '/dev/snd/controlC0', '/dev/snd/midiC0D0', '/dev/snd/pcmC0D1p', '/dev/snd/pcmC0D0c', '/dev/snd/pcmC0D0p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer': 'amixer'
Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer': 'amixer'
DistroRelease: Ubuntu 18.04
InstallationDate: Installed on 2020-11-09 (0 days ago)
InstallationMedia: Ubuntu-Server 18.04.5 LTS "Bionic Beaver" - Release amd64 (20200806.1)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
MachineType: VMware, Inc. VMware Virtual Platform
Package: linux (not installed)
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 svgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.4.0-52-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro maybe-ubiquity
ProcVersionSignature: Ubuntu 5.4.0-52.57~18.04.1-generic 5.4.65
RelatedPackageVersions:
 linux-restricted-modules-5.4.0-52-generic N/A
 linux-backports-modules-5.4.0-52-generic N/A
 linux-firmware 1.173.19
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic uec-images
Uname: Linux 5.4.0-52-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lxd plugdev sudo
_MarkForUpload: True
dmi.bios.date: 07/22/2020
dmi.bios.vendor: Phoenix Technologies LTD
dmi.bios.version: 6.00
dmi.board.name: 440BX Desktop Reference Platform
dmi.board.vendor: Intel Corporation
dmi.board.version: None
dmi.chassis.asset.tag: No Asset Tag
dmi.chassis.type: 1
dmi.chassis.vendor: No Enclosure
dmi.chassis.version: N/A
dmi.modalias: dmi:bvnPhoenixTechnologiesLTD:bvr6.00:bd07/22/2020:svnVMware,Inc.:pnVMwareVirtualPlatform:pvrNone:rvnIntelCorporation:rn440BXDesktopReferencePlatform:rvrNone:cvnNoEnclosure:ct1:cvrN/A:
dmi.product.name: VMware Virtual Platform
dmi.product.version: None
dmi.sys.vendor: VMware, Inc.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1903612

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

apport information

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

This problem is replicated on both virtual and physical machines.

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

​Linux 4.15 is a kernel module, and Linux 5.4 is a built-in module.
​I suspect it's a kernel build option issue.

[Po-Hsu Lin (cypressyew)]

I can confirm this issue exists.

Config difference on the Bionic tree:

[hwe-5.4]$ grep CONFIG_VFIO_PCI ubuntu-bionic/debian.hwe-5.4/config/config.common.ubuntu
CONFIG_VFIO_PCI_IGD=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_NVLINK2=y
CONFIG_VFIO_PCI_VGA=y
[hwe-5.4]$ grep CONFIG_VFIO_PCI ubuntu-bionic/debian.hwe-5.4/config/annotations
CONFIG_VFIO_PCI policy<{'amd64': 'y', 'arm64': 'm', 'armhf': 'm', 'i386': 'm', 'ppc64el': 'y', 's390x': 'm'}>
CONFIG_VFIO_PCI_VGA policy<{'amd64': 'y', 'i386': 'y'}>
CONFIG_VFIO_PCI_IGD policy<{'amd64': 'y', 'i386': 'y'}>
CONFIG_VFIO_PCI mark<ENFORCED>note<LP#1636733>

[master-next]$ grep CONFIG_VFIO_PCI ubuntu-bionic/debian.master/config/config.common.ubuntu
CONFIG_VFIO_PCI_IGD=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_VGA=y
[master-next]$ grep CONFIG_VFIO_PCI ubuntu-bionic/debian.master/config/annotations
CONFIG_VFIO_PCI policy<{'amd64': 'm', 'arm64': 'm', 'armhf': 'm', 'i386': 'm', 'ppc64el': 'y', 's390x': 'm'}>
CONFIG_VFIO_PCI_VGA policy<{'amd64': 'y', 'i386': 'y'}>
CONFIG_VFIO_PCI mark<ENFORCED>note<LP#1636733>

https://github.com/torvalds/linux/blob/master/drivers/vfio/pci/Kconfig

This will need to be invesitgated.

[Po-Hsu Lin (cypressyew)]

This vfio-pci was changed from module to built-in in 5.4 kernel for bug 1770845
With patch https://lists.ubuntu.com/archives/kernel-team/2019-December/106508.html

And it can be found in /lib/modules/$(uname -r)/modules.builtin
$ grep vfio /lib/modules/5.4.0-53-generic/modules.builtin
kernel/drivers/vfio/vfio.ko
kernel/drivers/vfio/vfio_virqfd.ko
kernel/drivers/vfio/vfio_iommu_type1.ko
kernel/drivers/vfio/pci/vfio-pci.ko
$ uname -a
Linux gummo 5.4.0-53-generic #59~18.04.1-Ubuntu SMP Wed Oct 21 12:14:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Do you have issue with using this vfio-pci on B-5.4 or Focal 5.4?
Thanks

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

Thanks Po-Hsu Lin.

The problem reappears in Bionic + HWE 5.4.
​Focal does not. ​It works fine.

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

For example...

```
ubuntu@DL360G9-4:~$ uname -a
Linux DL360G9-4 5.4.0-53-generic #59~18.04.1-Ubuntu SMP Wed Oct 21 12:14:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

ubuntu@DL360G9-4:~$ cat /etc/os-release |grep PRETTY_NAME
PRETTY_NAME="Ubuntu 18.04.5 LTS"

ubuntu@DL360G9-4:~$ modinfo vfio-pci
modinfo: ERROR: Module vfio-pci not found.

ubuntu@DL360G9-4:~$ grep vfio /lib/modules/5.4.0-53-generic/modules.builtin
kernel/drivers/vfio/vfio.ko
kernel/drivers/vfio/vfio_virqfd.ko
kernel/drivers/vfio/vfio_iommu_type1.ko
kernel/drivers/vfio/pci/vfio-pci.ko
```

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

And Here is..

```
ubuntu@DL360G9-4:~$ tree /lib/modules/5.4.0-53-generic/kernel/drivers/vfio
/lib/modules/5.4.0-53-generic/kernel/drivers/vfio
└── mdev
    ├── mdev.ko
    └── vfio_mdev.ko

1 directory, 2 files
```

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

​If this problem is not fixed, using DPDK on the Ubuntu HWE Kernel will be difficult.
​Is a correction planned?

[Po-Hsu Lin (cypressyew)]

Hello,

even with modinfo command not working with this module, it looks like the vfio-pci module is still being built and available with 5.4.0-53-generic #59~18.04.1-Ubuntu kernel:
$ ls /sys/module/vfio_pci/drivers/
pci:vfio-pci
$ uname -a
Linux gummo 5.4.0-53-generic #59~18.04.1-Ubuntu SMP Wed Oct 21 12:14:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

We will investigate why the modinfo command is not working.
Does the DPDK not working for you with this 5.4 kernel on Bionic?

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

​Bad News. ​Does not work with Bionic + HWE 5.4.0-54.

```
ubuntu@DL360G9-4:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.4.0-54-generic root=/dev/mapper/vgroot-lvroot ro iommu=pt intel_iommu=pt isocpus=2-11,14-23,26-35,38-47 default_hugepagesz=1G hugepagesz=1G hugepages=16

ubuntu@DL360G9-4:~$ sudo dpdk-devbind --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno1 drv=tg3 unused=vfio-pci *Active*
0000:02:00.1 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno2 drv=tg3 unused=vfio-pci
0000:02:00.2 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno3 drv=tg3 unused=vfio-pci
0000:02:00.3 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno4 drv=tg3 unused=vfio-pci
0000:08:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=ens1f0 drv=ixgbe unused=vfio-pci
0000:08:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=ens1f1 drv=ixgbe unused=vfio-pci
0000:08:10.0 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16 drv=ixgbevf unused=vfio-pci
0000:08:10.1 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f1 drv=ixgbevf unused=vfio-pci
0000:08:10.2 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f2 drv=ixgbevf unused=vfio-pci
0000:08:10.3 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f3 drv=ixgbevf unused=vfio-pci
0000:08:10.4 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f4 drv=ixgbevf unused=vfio-pci
0000:08:10.5 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f5 drv=ixgbevf unused=vfio-pci
0000:08:10.6 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f6 drv=ixgbevf unused=vfio-pci
0000:08:10.7 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s16f7 drv=ixgbevf unused=vfio-pci
0000:08:11.0 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17 drv=ixgbevf unused=vfio-pci
0000:08:11.1 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f1 drv=ixgbevf unused=vfio-pci
0000:08:11.2 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f2 drv=ixgbevf unused=vfio-pci
0000:08:11.3 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f3 drv=ixgbevf unused=vfio-pci
0000:08:11.4 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f4 drv=ixgbevf unused=vfio-pci
0000:08:11.5 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f5 drv=ixgbevf unused=vfio-pci
0000:08:11.6 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f6 drv=ixgbevf unused=vfio-pci
0000:08:11.7 '82599 Ethernet Controller Virtual Function 10ed' if=enp8s17f7 drv=ixgbevf unused=vfio-pci

Other Network devices
=====================
<none>

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other Crypto devices
====================
<none>

Eventdev devices using DPDK-compatible driver
=============================================
<none>

Eventdev devices using kernel driver
=========================...

[Po-Hsu Lin (cypressyew)]

Hello Youhei,
we have found the `modinfo` command (from the kmod package[1]) will need to be at least with version 27 (which is available in Focal) to support reading information from built-in modules.

To be more precise, with these commits:
https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/commit/?h=v27&id=60084cf1cb24dc3a257976c0d539a4c3ffda5db5
https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/commit/?h=v27&id=73eed2aba3cbe18bbda40f18f7bc32e8de1ec81f
https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/commit/?h=v27&id=b866b2165ae6cf18fb26b635f4a7ee57c0334731
https://git.kernel.org/pub/scm/utils/kernel/kmod/kmod.git/commit/?h=v27&id=e7e2cb61fa9f1db3429d91ef6accff549500d268

I have these changes backported here:
https://code.launchpad.net/~cypressyew/ubuntu/+source/kmod/+git/kmod/+ref/builtin-test

If you want to give this a try, you can clone this repo and do the following:
  git clone -b builtin-test https://git.launchpad.net/~cypressyew/ubuntu/+source/kmod
  sudo apt install gtk-doc-tools autoconf automake libtool
  cd kmod
  ./autogen.sh
  ./configure CFLAGS='-g -O2' --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --with-rootprefix= --with-rootlibdir=/lib
  make
  cd tools
  ./modinfo vfio-pci

This will fix this modinfo issue, it can print the built-in module info correctly. Hopefully we can push this to Ubuntu archive later.

Regarding DPDK, I have no experience with it, not sure if it will use the modinfo command here (a quick check with their repository[2] I didn't see it calling modinfo directly.)

I found this article with keyword "unused vfio-pci dpdk" on the Internet, it looks like you will have to bind the driver to the device with something like:
# dpdk-devbind -b vfio-pci 0000:01:00.0 0000:01:00.1
Hope this helps:
https://www.miraclelinux.com/tech-blog/gofc5x

[1] https://launchpad.net/ubuntu/+source/kmod
[2] https://github.com/DPDK/dpdk

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

Thank you for fixing the bug.
This bug has been closed, but I will try it this weekend.

[Youhei Tooyama (VirtualTech Japan) (ytooyama-virtualtech)]

Thanks Po-Hsu Lin.
The patch version of the kmod Package worked fine in my environment as well.

# cd ~/kmod/tools
# ./modinfo vfio-pci
name: vfio_pci
filename: (builtin)
description: VFIO PCI - User Level meta-driver
author: Alex Williamson <email address hidden>
license: GPL v2
version: 0.2
parm: ids:Initial PCI IDs to add to the vfio driver, format is "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and multiple comma separated entries can be specified (string)
parm: nointxmask:Disable support for PCI 2.3 style INTx masking. If this resolves problems for specific devices, report lspci -vvvxxx to <email address hidden> so the device can be fixed automatically via the broken_intx_masking flag. (bool)
parm: disable_vga:Disable VGA resource access through vfio-pci (bool)
parm: disable_idle_d3:Disable using the PCI D3 low power state for idle, unused devices (bool)

# dpdk-devbind --status

Network devices using DPDK-compatible driver
============================================
<none>

Network devices using kernel driver
===================================
0000:02:00.0 'NetXtreme BCM5717 Gigabit Ethernet PCIe 1665' if=eno1 drv=tg3 unused=vfio-pci *Active*
0000:02:00.1 'NetXtreme BCM5717 Gigabit Ethernet PCIe 1665' if=eno2 drv=tg3 unused=vfio-pci
0000:0e:00.0 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp14s0f0 drv=e1000e unused=vfio-pci
0000:0e:00.1 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp14s0f1 drv=e1000e unused=vfio-pci
0000:0f:00.0 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp15s0f0 drv=e1000e unused=vfio-pci
0000:0f:00.1 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp15s0f1 drv=e1000e unused=vfio-pci

# dpdk-devbind -b vfio-pci 0000:0f:00.0
# dpdk-devbind --status

Network devices using DPDK-compatible driver
============================================
0000:0f:00.0 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' drv=vfio-pci unused=e1000e

Network devices using kernel driver
===================================
0000:02:00.0 'NetXtreme BCM5717 Gigabit Ethernet PCIe 1665' if=eno1 drv=tg3 unused=vfio-pci *Active*
0000:02:00.1 'NetXtreme BCM5717 Gigabit Ethernet PCIe 1665' if=eno2 drv=tg3 unused=vfio-pci
0000:0e:00.0 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp14s0f0 drv=e1000e unused=vfio-pci
0000:0e:00.1 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp14s0f1 drv=e1000e unused=vfio-pci
0000:0f:00.1 '82571EB/82571GB Gigabit Ethernet Controller (Copper) 10bc' if=enp15s0f1 drv=e1000e unused=vfio-pci

Testing Environment:

# lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
# uname -a
Linux ml110gen9 5.4.0-77-generic #86~18.04.1-Ubuntu SMP Fri Jun 18 01:23:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
# apt list -a dpdk
Listing... Done
dpdk/bionic-updates,now 17.11.10-0ubuntu0.1 amd64 [installed]

[Jun Shiozawa (junhair)]

Hello,Po-Hsu Lin

$ uname -r
5.11.0-27-generic

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal

$ sudo vi /etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
kvm
kvm_intel

:wq

Reboot Ubuntu.

$ lsmod | grep vfio

empty...

$ lsmod | grep kvm

kvm_intel 294912 0
kvm 819200 1 kvm_intel

$ modinfo vfio
name: vfio
filename: (builtin)
softdep: post: vfio_iommu_type1 vfio_iommu_spapr_tce
alias: devname:vfio/vfio
alias: char-major-10-196
description: VFIO - User Level meta-driver
author: Alex Williamson <email address hidden>
license: GPL v2
file: drivers/vfio/vfio
version: 0.3
parm: enable_unsafe_noiommu_mode:Enable UNSAFE, no-IOMMU mode. This mode provides no device isolation, no DMA translation, no host kernel protection, cannot be used for device assignment to virtual machines, requires RAWIO permissions, and will taint the kernel. If you do not know what this is for, step away. (default: false) (bool)

$ modinfo vfio_iommu_type1
name: vfio_iommu_type1
filename: (builtin)
description: Type1 IOMMU driver for VFIO
author: Alex Williamson <email address hidden>
license: GPL v2
file: drivers/vfio/vfio_iommu_type1
version: 0.2
parm: allow_unsafe_interrupts:Enable VFIO IOMMU support for on platforms without interrupt remapping support. (bool)
parm: disable_hugepages:Disable VFIO IOMMU support for IOMMU hugepages. (bool)
parm: dma_entry_limit:Maximum number of user DMA mappings per container (65535). (uint)

$ modinfo vfio_pci
name: vfio_pci
filename: (builtin)
description: VFIO PCI - User Level meta-driver
author: Alex Williamson <email address hidden>
license: GPL v2
file: drivers/vfio/pci/vfio-pci
version: 0.2
parm: ids:Initial PCI IDs to add to the vfio driver, format is "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and multiple comma separated entries can be specified (string)
parm: nointxmask:Disable support for PCI 2.3 style INTx masking. If this resolves problems for specific devices, report lspci -vvvxxx to <email address hidden> so the device can be fixed automatically via the broken_intx_masking flag. (bool)
parm: disable_vga:Disable VGA resource access through vfio-pci (bool)
parm: disable_idle_d3:Disable using the PCI D3 low power state for idle, unused devices (bool)
parm: enable_sriov:Enable support for SR-IOV configuration. Enabling SR-IOV on a PF typically requires support of the userspace PF driver, enabling VFs without such support may result in non-functional VFs or PF. (bool)
parm: disable_denylist:Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users. (bool)

$ modinfo vfio_virqfd
name: vfio_virqfd
filename: (builtin)
description: IRQFD suppo...