Improper globbing in rules for /sys/devices PCI paths

[Keifer Snedeker]

[ Impact ]

* AppArmor profiles with rules for /sys/devices/ PCI paths using decimal patterns may miss some PCI devices whose PCI domain or bus starts with or contains hex characters a-f.

* Example: On a DGX-2 with 10 nvme drives provisioned with Plucky, the output of lsblk is missing entries for nvme2 (pci0000:ae), nvme3 (pci0000:ae), nvme8 (pci0000:d7), and nvme9 (pci0000:d7), and there are apparmor="DENIED" messages for each omitted drive.

* This was addressed by updating any profile containing rule(s) for /sys/devices/ PCI domain:bus paths to use hex matching instead of dec. The alias `@{pci_bus}=pci@{hex4}:@{hex2}` was established and used in all such cases (e.g., `@{sys}/devices/pci[0-9]*:[0-9]*/...` -> `@{sys}/devices/@{pci_bus}/...`) to standardize the globbing used against /sys/devices PCI paths.

[ Test Plan ]

* On a test device provisioned with Plucky which has one or more block devices on PCI whose PCI domain or bus begins with a hex character a-f (e.g., a device with many NVME drives), invoke `lsblk` and compare the output to what is expected for the hardware physically present on the device (or temporarily disable AppArmor and compare the output of `lsblk`).

After installing the proposed packages, invoke `lsblk` again, and compare the listed block devices (none should be missing in this case). Note that this example is specific to Plucky as the lsblk profile was not present in releases prior.

* As many of these profiles are related to software associated with a specific set of hardware, I am unable to manually test the changes across all affected profiles. However, since the changes are uniform across the affected profiles, the implications of a `@{sys}/devices/pci[0-9]*:[0-9]*/...` rule being updated to `@{sys}/devices/@{pci_bus}/` in one profile should not differ from the implications of the same change in another.

[ Where problems could occur ]

* Improper matching of /sys/devices/ PCI paths may result in devices being 'invisible' to the software for which the profile(s) regulate (such is the case prior to this patch).

* As far as I'm aware, any PCI bus:domain listed in /sys/devices/ is strictly in the format pciHHHH:HH (where 'H' are hex chars [0-9a-f]). If there are *any* exceptions to this which I'm unaware of, devices on these bus:domains may be omitted by these rules.

[ Other Info ]

Upstream MR (merged): https://gitlab.com/apparmor/apparmor/-/merge_requests/1725
[1] https://launchpad.net/~ks0/+archive/ubuntu/plucky-apparmor-pci-patch-2
[2] https://launchpad.net/~ks0/+archive/ubuntu/noble-apparmor-pci-patch-3
[3] https://launchpad.net/~ks0/+archive/ubuntu/jammy-apparmor-pci-patch-3

[ Verbose Example with `lsblk` ]

Expected to see all nvmeXn1 (0-9) devices listed, but some are omitted, such as nvme2n1. nvme2n1 appears under the PCI directory pci0000:ae, thus AppArmor denials appear in dmesg and nvme2n1 is omitted from the output of lsblk (the same applies for the other ommitted devices: nvme3n1, nvme5n1, nvme7n1).

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 1 29.3G 0 disk
└─sda1 8:1 1 29.3G 0 part
sdb 8:16 1 0B 0 disk
sr0 11:0 1 1024M 0 rom
nvme1n1 259:0 0 894.3G 0 disk
├─nvme1n1p1 259:2 0 512M 0 part /boot/efi
└─nvme1n1p2 259:3 0 893.8G 0 part /
nvme0n1 259:1 0 894.3G 0 disk
nvme4n1 259:4 0 3.5T 0 disk
nvme9n1 259:6 0 3.5T 0 disk
nvme8n1 259:8 0 3.5T 0 disk
nvme6n1 259:11 0 3.5T 0 disk

$ readlink -f /sys/class/block/nvme2n1/device
/sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/0000:b0:00.0/0000:b1:00.0/nvme/nvme2

$ sudo dmesg | grep -i nvme
...
[11748.808896] audit: type=1400 audit(1750465699.990:180): apparmor="DENIED" operation="open" class="file" profile="lsblk" name="/sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/0000:b0:00.0/0000:b1:00.0/nvme/nvme2/nvme2n1/hidden" pid=3734 comm="lsblk" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
[11748.808904] audit: type=1400 audit(1750465699.990:181): apparmor="DENIED" operation="open" class="file" profile="lsblk" name="/sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/0000:b0:00.0/0000:b1:00.0/nvme/nvme2/nvme2n1/dev" pid=3734 comm="lsblk" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0
[11748.808931] audit: type=1400 audit(1750465699.990:182): apparmor="DENIED" operation="open" class="file" profile="lsblk" name="/sys/devices/pci0000:ae/0000:ae:00.0/0000:af:00.0/0000:b0:00.0/0000:b1:00.0/nvme/nvme2/dev" pid=3734 comm="lsblk" requested_mask="r" denied_mask="r" fsuid=1000 ouid=0

[Ryan Lee (rlee287)]

Jammy and Noble do not have an lsblk profile, but they do have PCI based rules and abstractions that have the same issue.

[Keifer Snedeker (ks0)]

@rlee287 Sorry for the confusion. The scope of the original MR and bug were extended beyond just the lsblk profile as the same improper matching for PCI device paths is present in many other profiles (link below). I will update the bug title and description here shortly.

https://gitlab.com/apparmor/apparmor/-/merge_requests/1725

[Ryan Lee (rlee287)]

Did you remove Questing because the bug has been fixed or because the bug was never there in the first place? If the bug was there but is now fixed, apparmor (Ubuntu Questing) should still be targeted but with its status set to "Fix Released" instead.

[Keifer Snedeker (ks0)]

@rlee287 I asked Jacob to remove Questing as the commit containing the necessary changes is now in Questing due to recent syncs with upstream. Should it still be targeted to Questing with status "Fix Released" in this case?

[Chris Halse Rogers (raof)]

The changes here look reasonable. However, the test case should be amended to not suggest the use of a PPA - tests for SRUs need to be done against the packages we're releasing! (I have done this)

[Chris Halse Rogers (raof)]

Hello Keifer, or anyone else affected,

Accepted apparmor into plucky-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apparmor/4.1.0~beta5-0ubuntu14.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-plucky to verification-done-plucky. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-plucky. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apparmor (4.1.0~beta5-0ubuntu14.2) for plucky have finished running.
The following regressions have been reported in tests triggered by the package:

dbus-broker/36-1ubuntu0.25.04.1 (s390x)
dovecot/1:2.3.21.1+dfsg1-1ubuntu2 (amd64)
systemd/257.4-1ubuntu3.1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/plucky/update_excuses.html#apparmor

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Keifer Snedeker (ks0)]

I tested 4.1.0~beta5-0ubuntu14.2 from plucky-proposed and verified it fixes the issue - as compared to the previous version of apparmor, with 4.1.0~beta5-0ubuntu14.2 installed the output of `lsblk` is no longer missing drives, and there are no apparmor="DENIED" in dmesg.

The autopkgtest regressions listed did not appear to be related to the package(s) themselves, and were resolved after submitting autopkgtest retries for each.

[Julian Andres Klode (juliank)]

The verification of the Stable Release Update for apparmor has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 4.1.0~beta5-0ubuntu14.2

---------------
apparmor (4.1.0~beta5-0ubuntu14.2) plucky; urgency=medium

  * profiles: make /sys/devices PCI paths hex-aware (LP: #2115234)

 -- Keifer Snedeker <email address hidden> Tue, 09 Sep 2025 17:23:48 -0400

[Julian Andres Klode (juliank)]

Hello Keifer, or anyone else affected,

Accepted apparmor into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apparmor/4.0.1really4.0.1-0ubuntu0.24.04.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Julian Andres Klode (juliank)]

Hello Keifer, or anyone else affected,

Accepted apparmor into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apparmor/3.0.4-2ubuntu2.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apparmor (4.0.1really4.0.1-0ubuntu0.24.04.5) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

dbus-broker/35-2ubuntu0.1 (s390x)
libreoffice/4:24.2.7-0ubuntu0.24.04.4 (amd64, arm64, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#apparmor

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted apparmor (3.0.4-2ubuntu2.5) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

firejail/0.9.66-2 (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#apparmor

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Keifer Snedeker (ks0)]

A couple of the autopkgtests marked 'regression' don't appear to be indicative of true regressions -

1.
apparmor/firejail/jammy/s390x
https://autopkgtest.ubuntu.com/packages/f/firejail/jammy/s390x
https://autopkgtest.ubuntu.com/results/autopkgtest-jammy/jammy/s390x/f/firejail/20251014_154207_c54b0@/log.gz

Failing due to a missing dependency (depends on firefox, but firefox isn't available for jammy/s390x) - entire history of tests in this queue seem to be failing for the same reason.

1009s The following packages have unmet dependencies:
1009s satisfy:command-line : Depends: firefox but it is not installable
1009s E: Unable to correct problems, you have held broken packages.

https://packages.ubuntu.com/jammy/firefox - not available for jammy/s390x

2.
apparmor/dbug-broker/noble/s390x
https://autopkgtest.ubuntu.com/packages/d/dbus-broker/noble/s390x

Two different failure causes have been observed here -
a.
https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/d/dbus-broker/20251022_163451_3e47f@/log.gz

VM's clock was behind the source files' timestamps. This was observed only once, on the most recent failure.

103s ERROR: Clock skew detected. File /tmp/autopkgtest.Fjq5ev/build.n3V/src/obj-s390x-linux-gnu/../meson.build has a time stamp 58.6022s in the future.

b.
https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/d/dbus-broker/20251014_134942_c3e7c@/log.gz

On all other re-runs of this, compilation of test executables fails due to 'DEBUGINFOD_URLS' not being defined. The number of test executables which fail to compile due to this varies (i.e., it is not consistently the same N test executables failing to compile due to this on each re-run, which is confusing). This seems to align with cause of historical failures in the queue (though there are eventual passes).

244s cc: fatal error: environment variable ‘DEBUGINFOD_URLS’ not defined

---

(1) seems like it should be resolved with a bad-test hint or made passing with a migration-reference/0 run. (2) also seems very unlikely to be related to any of the apparmor changes being tested, though I am unsure of the root cause or what the solution should be to unblock this SRU.

Any thoughts or recommendations for next steps?

[Mitchell Augustin (mitchellaugustin)]

The tests that Keifer mentioned in https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2115234/comments/15 which were failing also failed in the same way with a migration-reference/0 run.