apparmor autotest failure on jammy with linux 5.15

[Andrea Righi]

[Impact]

test-aa-notify is also checking if the output of `aa-notify --help` matches a specific text. However it looks like this output has changed in jammy so the autopkgtest is reporting errors like this:

05:17:31 ERROR| [stderr] === test-aa-notify.py ===
05:17:31 ERROR| [stderr] .ssF.
05:17:31 ERROR| [stderr] ======================================================================
05:17:31 ERROR| [stderr] FAIL: test_help_contents (__main__.AANotifyTest)
05:17:31 ERROR| [stderr] Test output of help text
05:17:31 ERROR| [stderr] ----------------------------------------------------------------------
05:17:31 ERROR| [stderr] Traceback (most recent call last):
05:17:31 ERROR| [stderr] File "/tmp/testlibmse00lib/source/jammy/apparmor-3.0.3/utils/test/test-aa-notify.py", line 178, in test_help_contents
05:17:31 ERROR| [stderr] self.assertEqual(expected_output_is, output, result + output)
05:17:31 ERROR| [stderr] AssertionError: 'usag[189 chars]ptional arguments:\n -h, --help sh[746 chars]de\n' != 'usag[189 chars]ptions:\n -h, --help show this hel[735 chars]de\n'
05:17:31 ERROR| [stderr] usage: aa-notify [-h] [-p] [--display DISPLAY] [-f FILE] [-l] [-s NUM] [-v]
05:17:31 ERROR| [stderr] [-u USER] [-w NUM] [--debug]
05:17:31 ERROR| [stderr]
05:17:31 ERROR| [stderr] Display AppArmor notifications or messages for DENIED entries.
05:17:31 ERROR| [stderr]
05:17:31 ERROR| [stderr] - optional arguments:
05:17:31 ERROR| [stderr] + options:
05:17:31 ERROR| [stderr] -h, --help show this help message and exit
05:17:31 ERROR| [stderr] -p, --poll poll AppArmor logs and display notifications
05:17:31 ERROR| [stderr] --display DISPLAY set the DISPLAY environment variable (might be needed if
05:17:31 ERROR| [stderr] sudo resets $DISPLAY)
05:17:31 ERROR| [stderr] -f FILE, --file FILE search FILE for AppArmor messages
05:17:31 ERROR| [stderr] -l, --since-last display stats since last login
05:17:31 ERROR| [stderr] -s NUM, --since-days NUM
05:17:31 ERROR| [stderr] show stats for last NUM days (can be used alone or with
05:17:31 ERROR| [stderr] -p)
05:17:31 ERROR| [stderr] -v, --verbose show messages with stats
05:17:31 ERROR| [stderr] -u USER, --user USER user to drop privileges to when not using sudo
05:17:31 ERROR| [stderr] -w NUM, --wait NUM wait NUM seconds before displaying notifications (with
05:17:31 ERROR| [stderr] -p)
05:17:31 ERROR| [stderr] --debug debug mode
05:17:31 ERROR| [stderr] : Got output "usage: aa-notify [-h] [-p] [--display DISPLAY] [-f FILE] [-l] [-s NUM] [-v]
05:17:31 ERROR| [stderr] [-u USER] [-w NUM] [--debug]

[Test case]

Simply run test-aa-notify.py from the autopkgtests.

[Fix]

Update the expected output returned by `aa-notify --help` in test-aa-notify.py.

[Regression potential]

This is just an autopkgtest, we may see regressions if the test is used with older version of apparmor-notify. With newer versions there's no risk of regressions.

[Andrea Righi (arighi)]

Simple fix in the attached debdiff

[Christian Boltz (cboltz)]

This was already fixed upstream with https://gitlab.com/apparmor/apparmor/-/merge_requests/848 (with a slightly different patch that works for all python versions).

AppArmor >= 3.0.5 will include the fix.

[Alex Murray (alexmurray)]

FYI I am working on merging apparmor-3.0.4 from debian unstable to jammy at the moment which should resolve this.

[Andrea Righi (arighi)]

@alexmurray thanks for the update! BTW I found another issue with test-network.py: this test is failing because utils/apparmor/rule/network.py is missing the mctp protocol (in network_domain_keywords[]).

I checked upstream, but I couldn't find any fix for this, do you want me to open another tracking bug / send a patch?

[John Johansen (jjohansen)]

@arighi: mctp is already supported in the 3.0.4 release that @alexmurray is working on merging

[John Johansen (jjohansen)]

I should note that mctp is NOT part of the abi change in 3.0.4. This means by default mctp mediation will not be enforced by policy. It will be accepted in rules if present but since policy was not developed with mctp in mind, turning it on by default for the policy would be an abi break and could cause some applications to fail unexpectedly.

To have mctp mediation enforced it needs to be added to the abi file. Or profiles that should have it enforced need to change their abi file to one that supports mctp.

[Andrea Righi (arighi)]

updated debdiff in attach FWIW

[Dimitri John Ledkov (xnox)]

@alexmurray @jjohansen

When are updated apparmor going to be upload that continues to pass existing test-suites / adt?

At this point failing apparmor ADT, blocks releasing all kernels in jammy, preventing development of all kernels, and prevents security kernel fixes.

To unblock kernel development we need apparmor to never fail ADT testing in devel series, as new kernel is developed. We do not want to hint to ignore it, because we must never regress apparmor.

Is it ok to upload the debdiff from #7 right away? Because this bug cannot wait for new upstream release of apparmor getting integrated in Ubuntu and migrating. 3 days for test-suite only fixes is too long.

[Dimitri John Ledkov (xnox)]

Ah, it is president's day & night time in australia.

I will upload this, to unblock releasing jammy kernels. And we can revisit this once everyone is back to back this out; or get a different implementation in.

Blocking kernel testing with app armor test suite is developer time critical, and prevents multiple teams from working on the next kernel.

[John Johansen (jjohansen)]

"To unblock kernel development we need apparmor to never fail ADT testing in devel series, as new kernel is developed. We do not want to hint to ignore it, because we must never regress apparmor."

unfortunately this is just not possible with the way kernel development works. The addition of new "features" will break apparmor if there is any support in the kernel for it as apparmor is default deny. There are also other reasons kernel changes may result in test failures.

The only way to never block would be to ignore failures on the devel series, but as you noted we don't want to regress either. Its a tough situation, I don't have a good solution.

"Is it ok to upload the debdiff from #7 right away? Because this bug cannot wait for new upstream release of apparmor getting integrated in Ubuntu and migrating. 3 days for test-suite only fixes is too long."

Unfortunately it is NOT just a test suite issue. This requires an update to the the policy compiler.

@alexmurray is currently planning to upload the latest version tomorrow (his tomorrow, which is in just a few hours), but as you note it will then take time to migrate.

[Alex Murray (alexmurray)]

FYI I am preparing this in https://bileto.ubuntu.com/#/ticket/4796 - I have included the original patch from arighi to fix the aa-notify tests too. Once britney looks happy with this I will upload it to jammy-proposed.

[John Johansen (jjohansen)]

sorry, I was confused a bit about the issue. I have no objection to uploading the diff from #7. Still while the patch makes the immediate mctp issue go away from the current tests it isn't a full fix.

[Alex Murray (alexmurray)]

Hmm so had to redo my merge after the 3.0.3-0ubuntu9 upload... see new bileto ticket/PPA for the current version of it https://bileto.ubuntu.com/#/ticket/4797

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 3.0.4-1ubuntu1

---------------
apparmor (3.0.4-1ubuntu1) jammy; urgency=medium

  * Merge from Debian unstable; remaining changes:
    - Drop the following patches that have been included in the upstream
      release or which Debian has also included:
      - d/p/ubuntu/adjust-for-ibus-1.5.22.patch
      - d/p/ubuntu/0011-add-mctp-network-protocol.patch
    - Refresh
      d/p/regression-tests-fix-aa_policy_cache-when-using-syst.patch to the
      official version from upstream
    - d/p/u/samba-systemd-interaction.patch: allow smbd to interact with
      systemd
    - d/p/u/libnss-systemd.patch: allow accessing the libnss-systemd
      VarLink sockets and DBus APIs
    - Disable lto builds
    - Fix autotest test-aa-notify.py
      - d/p/ubuntu/fix-test-aa-notify.patch
    - Drop outdated lintian-overrides

apparmor (3.0.4-1) unstable; urgency=medium

  * New upstream release
  * apparmor-profiles: install new samba-bgqd profile
  * Drop backported patches that are now obsolete
  * debian/allow-access-to-ibus-socket.patch: drop support for pre-Bullseye
    ibus path
  * Declare compliance with Policy 4.6.0.1
  * Drop XS- prefix for adopted Python-Version control field
  * Add new symbols

apparmor (3.0.3-6) unstable; urgency=medium

  * debian/rules: let "set -e" take effect (Closes: #998843)
  * Add support for Python 3.10 (Closes: #998686):
    - upstream-ab4cfb5e-replace-distutils-with-setuptools.patch: new patch,
      edited to drop changes to upstream .gitignore.
    - Add build-dependency on python3-setuptools

apparmor (3.0.3-5) unstable; urgency=medium

  [ Debian Janitor ]
  * Remove constraints unnecessary since stretch.

  [ Helmut Grohne ]
  * Make the package cross-buildable (Closes: #984582):
    - Multiarchify python Build-Depends
    - Let dh_auto_build pass cross tools to make
    - Annotate perl build-dependency with !nocheck

  [ intrigeri ]
  * Remove obsolete libapparmor-perl on upgrade

apparmor (3.0.3-4) unstable; urgency=medium

  * Merge apparmor-easyprof into apparmor-utils (Closes: #972880)
  * Make apparmor-utils and python3-apparmor arch:all (Closes: #972881)

apparmor (3.0.3-3) unstable; urgency=medium

  * Adjust gbp.conf and Vcs-* control fields for 3.0.x now being in sid.
  * Stop building the libapparmor-perl binary package (Closes: #993565)
  * Update Lintian overrides
  * Add B-D on dh-sequence-python3, to workaround #996089 in Lintian
  * B-D: python3-all → python3-all:any, to appease Lintian

apparmor (3.0.3-2) unstable; urgency=medium

  * Upload to unstable

apparmor (3.0.3-1) experimental; urgency=medium

  * New upstream release
  * Drop debian/Revert-libapparmor-fixing-setup.py-call-when-crosscompili.patch:
    obsolete
  * Refresh patches
  * Merge changes from sid, up to 2.13.6-10
  * upstream-6cfc6eee-python-3.10.patch: new patch,
    for compatibility with Python 3.10

 -- Alex Murray <email address hidden> Tue, 22 Feb 2022 10:13:44 +1030