Enable swiotlb to avoid untrusted devices errors on AMD platforms

[You-Sheng Yang]

It's been observed that plugging in a TBT3 NVME device to a port marked
with ExternalFacingPort that some DMA transactions occur that are not a
full page and so the DMA API attempts to use software bounce buffers
instead of relying upon the IOMMU translation.

This doesn't work and leads to messaging like:

swiotlb buffer is full (sz: 4096 bytes), total 0 (slots), used 0 (slots)

The bounce buffers were originally set up, but torn down during
the boot process.
* This happens because as part of IOMMU initialization
  `amd_iommu_init_dma_ops` gets called and resets the global swiotlb to 0.
* When late_init gets called `pci_swiotlb_late_init` `swiotlb_exit` is
  called and the buffers are torn down.

This can be observed in the logs:
```
[ 0.407286] AMD-Vi: Extended features (0x246577efa2254afa): PPR NX GT [5] IA GA PC GA_vAPIC
[ 0.407291] AMD-Vi: Interrupt remapping enabled
[ 0.407292] AMD-Vi: Virtual APIC enabled
[ 0.407872] software IO TLB: tearing down default memory pool
```
This series fixes the behavior of AMD IOMMU to enable swiotlb so that
non-page aligned DMA goes through a bounce buffer.

It also adds a message to help with debugging similar problems in the
future.

[You-Sheng Yang (vicamo)]

This affects kernel v5.11 and on.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1969086

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[You-Sheng Yang (vicamo)]

In https://launchpad.net/~canonical-hwe-team/+archive/ubuntu/intermediate-kernel starting from:
* linux-unstable/jammy version 5.17.0-9008.8+exp.13
* linux-oem-5.17/jammy version 5.17.0-9003.3+exp.9
* linux/jammy version 5.15.0-9027.28+exp.23

oem-5.14 and older to be decided due to https://bugs.launchpad.net/ubuntu/+source/linux-oem-5.14/+bug/1969091/comments/3.

[Timo Aaltonen (tjaalton)]

Fixed in 5.19 and up

[Timo Aaltonen (tjaalton)]

@vicamo, what about jammy/oem-5.17?

[Mario Limonciello (superm1)]

As dependency bug 1969091 was covered by v5.15.33 this is really just two patches, and the more important one is 5.15.y already. The other one is just messaging: f316ba0a8814f4c91e80a435da3421baf0ddd24c.

# git describe --contains 9e53c25f326604c838d528b430b8e603944b9e28
v5.15.46~190

It's landed in ubuntu's Jammy kernel too and OEM kernel too:
https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/commit/drivers/iommu/amd?h=master-next&id=a0468da388126aece4c989eee5a6cd83945cf731
https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-oem/+git/jammy/commit/drivers/iommu/amd?h=oem-5.17-next&id=6f64d8b2cb741bb9938fedc0446f1d8be6724ae6

$ git describe --contains a0468da388126aece4c989eee5a6cd83945cf731
Ubuntu-5.15.0-47.50~244