[UBUNTU 21.04] vfio: pass DMA availability information to userspace
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu on IBM z Systems |
Fix Released
|
Undecided
|
Skipper Bug Screeners | ||
linux (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Frank Heimes | ||
Groovy |
Fix Released
|
Undecided
|
Frank Heimes | ||
Hirsute |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
Description: vfio: pass DMA availability information to userspace
Symptom: vfio-pci device on s390 enters error state
Problem: Commit 492855939bdb added a limit to the number of concurrent
DMA requests for a vfio container. However, lazy unmapping in
s390 can in fact cause quite a large number of outstanding DMA
in qemu such as 'VFIO_MAP_DMA failed: No space left on device'
Solution: The solution requires a change to both kernel and qemu - For
the kernel, add the ability to provide the number of allowable
DMA requests via the VFIO_IOMMU_GET_INFO ioctl.
Reproduction: Put a vfio-pci device on s390 under I/O load
Upstream-ID: a717072007e8aed
Need also to be integrated into 20.10 and 20.04.
OK, just to clarify we don't need to fix bionic for this one, but rather focal (20.04) and groovy (20.10). Furthermore, for 20.04, 20.10 and 21.04 ONLY commit 7d6e1329652ed97
__________
SRU Justification:
[Impact]
* In case a vfio-pci device on s390x is under I/O load, vfio-pci device may end up in error state.
* The commit 492855939bdb added a limit to the number of concurrent DMA requests for a vfio container.
* However, lazy unmapping in s390x can in fact cause quite a large number of outstanding DMA requests
to build up prior to being purged - potentially the entire guest DMA space.
* This results in unexpected errors seen in qemu such as 'VFIO_MAP_DMA failed: No space left on device'.
* The solution requires a change to both kernel and qemu.
* The kernel side of things is addressed by this SRU.
* The fix adds the ability to provide the number of allowable DMA requests via VFIO_IOMMU_GET_INFO ioctl.
* The actual fix comes with commit 7d6e1329652e, but another fix ccd59dce1a21 is needed to get it cleanly applied.
[Fix]
* ccd59dce1a21f47
* 7d6e1329652ed97
[Test Case]
* IBM Z or LinuxONE hardware with Ubuntu Server 20.10 installed.
* PCIe adapters in place that provide vfio, like RoCE Express 2.
* A KVM host needs to be setup and a KVM guest (use again 20.10) that uses vfio.
* Generate I/O that flows through the vf and watch out for error like 'VFIO_MAP_DMA failed: No space left on device' in the log.
[Regression Potential]
* The first patch ccd59dce1a21 modifies the common code file drivers/
* But the reason is not that it introduces a lot of new things, it's a refactoring patch.
* Nevertheless if done in a bad way it can significantly harm the IO memory management of virtual function adapters.
* In worst case it may break them entirely, instead of 'just' exeeding the entire DMA space.
* Things could also go wrong while doing the mapping and unmapping of DMA, that may even have an impact beyond vf adapters - harming other DMA devices.
* The handling of dirty pages is also touched and the ioctl itself - which is important to keep the control of the devices.
* But as said before, it re-factoring work, it's upstream accepted since 5.9 and the provenance shows that many engineers had an eye on these changes.
* The second patch 7d6e1329652e - that inclides the needed fix - comes with far less modifications.
* It also tounches drivers/
* That allows to provide the current allowable number of DMA mappings to the userspace via the IOMMU info chain,
so that the userspace can take appropriate mitigation.
* Potential problems here can be that the current allowable number of DMA mappings are wrong and in best case just mappings are wasted and in worst case there are more reported than available in reality, which could have a severe impact.
* What happens in such a case is a bit depending on the userspace.
* This patch got upstream accepted with kernel 5.10 and pre-tested by IBM.
* In addition a PPA was created with a patched groovy kernel that was shared for further testing.
[Other]
* The patch got upstream accepted with kernel v5.10, hence it will land in Hirsute once the target kernel 5.10 is in.
* For 5.4 this will come as upstream stable update 5.4.90/91.
* For 5.8 (and 5.9) upstream stable support already ended, hence this SRU is only needed for groovy.
CVE References
tags: | added: architecture-s39064 bugnameltc-190211 severity-high targetmilestone-inin2104 |
Changed in ubuntu: | |
assignee: | nobody → Skipper Bug Screeners (skipper-screen-team) |
affects: | ubuntu → linux (Ubuntu) |
Changed in linux (Ubuntu Hirsute): | |
status: | In Progress → Fix Committed |
Changed in linux (Ubuntu Focal): | |
status: | Incomplete → Confirmed |
Changed in ubuntu-z-systems: | |
status: | Incomplete → Confirmed |
Changed in linux (Ubuntu Groovy): | |
status: | In Progress → Fix Committed |
Changed in linux (Ubuntu Focal): | |
status: | In Progress → Fix Committed |
Changed in ubuntu-z-systems: | |
status: | In Progress → Fix Committed |
The patch is in Ubuntu UNSTABLE (that will likely become the kernel for hirsute) as: 5.10-5. 10.0-0. 1
7d6e1329652e "vfio iommu: Add dma available capability"
starting with:
Ubuntu-
Hence updating Hirsute entry to 'In Progress'.