test-pivot_root_parse.py in apparmor failed with ImportError: No module named apparmor.aa

[Po-Hsu Lin]

Steps to reproduce:
# Via qa-regression-testing
git clone --depth=1 git://git.launchpad.net/qa-regression-testing
cd qa-regression-testing/scripts
sudo python3 ./test-apparmor.py -v ApparmorTestsuites.test_utils_testsuite

== Update base on John's comment in #1
Note that we cannot use apparmor cloned from launchpad directly, it requires some other local patches in qa-regression-testing:
https://git.launchpad.net/qa-regression-testing/tree/scripts/apparmor/patches
To experiment with the patched version via qa-regression-testing, comment out the following lines (1583, 1584) in tearDown():
        if os.path.exists(self.tmpdir):
            testlib.recursive_rm(self.tmpdir)
And run:
sudo python3 ./test-apparmor.py -v ApparmorTestsuites.test_utils_testsuite

The patched apparmor can be found in /tmp, now it can be tested with things like:
cd /tmp/testlibwl29iqzr/source/focal/apparmor-2.13.3/utils
export USE_SYSTEM=1
make check
== End of update

Output:
LANG=C podchecker -warning -warning *.pod
aa-audit.pod pod syntax OK.
aa-autodep.pod pod syntax OK.
aa-cleanprof.pod pod syntax OK.
aa-complain.pod pod syntax OK.
aa-decode.pod pod syntax OK.
aa-disable.pod pod syntax OK.
aa-easyprof.pod pod syntax OK.
aa-enforce.pod pod syntax OK.
aa-genprof.pod pod syntax OK.
aa-logprof.pod pod syntax OK.
aa-mergeprof.pod pod syntax OK.
aa-notify.pod pod syntax OK.
aa-remove-unknown.pod pod syntax OK.
aa-sandbox.pod pod syntax OK.
aa-status.pod pod syntax OK.
aa-unconfined.pod pod syntax OK.
logprof.conf.pod pod syntax OK.
aa-notify syntax OK
Checking aa-easyprof
Checking aa-genprof
Checking aa-logprof
Checking aa-cleanprof
Checking aa-mergeprof
Checking aa-autodep
Checking aa-audit
Checking aa-complain
Checking aa-enforce
Checking aa-disable
Checking aa-status
Checking aa-unconfined
Checking apparmor
Checking test/common_test.py
Checking test/minitools_test.py
Checking test/test-aa-decode.py
Checking test/test-aa-easyprof.py
Checking test/test-aa.py
Checking test/test-aamode.py
Checking test/test-aare.py
Checking test/test-baserule.py
Checking test/test-capability.py
Checking test/test-change_profile.py
Checking test/test-common.py
Checking test/test-config.py
Checking test/test-dbus.py
Checking test/test-example.py
Checking test/test-file.py
Checking test/test-libapparmor-test_multi.py
Checking test/test-logparser.py
Checking test/test-mount_parse.py
Checking test/test-network.py
Checking test/test-parser-simple-tests.py
Checking test/test-pivot_root_parse.py
Checking test/test-profile-list.py
Checking test/test-profile-storage.py
Checking test/test-ptrace.py
Checking test/test-regex_matches.py
Checking test/test-rlimit.py
Checking test/test-severity.py
Checking test/test-signal.py
Checking test/test-translations.py
Checking test/test-unix_parse.py
make[1]: Entering directory '/home/ubuntu/testlibwl29iqzr/source/focal/apparmor-2.13.3/utils/test'

=== test-example.py ===
.......
----------------------------------------------------------------------
Ran 7 tests in 0.000s

OK
.......
----------------------------------------------------------------------
Ran 7 tests in 0.000s

OK

=== test-pivot_root_parse.py ===
Traceback (most recent call last):
  File "test-pivot_root_parse.py", line 12, in <module>
    import apparmor.aa as aa
ImportError: No module named apparmor.aa
make[1]: *** [Makefile:81: check] Error 1
make[1]: Leaving directory '/home/ubuntu/testlibwl29iqzr/source/focal/apparmor-2.13.3/utils/test'
make: *** [Makefile:97: check] Error 2

Issue found on B-ibm-gt-5.4, F-oem-5.14, F-aws-fips after we changed how we run the ubuntu_qrt_apparmor test in bug 1982439.

Sub-test in ApparmorTestsuites.test_utils_testsuite will fail with this.

After some inspection it is because the script is trying to use python2. This can be verified with:
python2 -c "import apparmor.aa"
python3 -c "import apparmor.aa"

This appears to be a known issue.
https://gitlab.com/apparmor/apparmor/-/commit/af69088546d7d2f7188b9dec814f81721254964f

And this patch has already landed in apparmor on Jammy, we will need this for Bionic / Focal as well. A temporary workaround is to run:
export PYTHON_VERSIONS=python3

Before starting the tests.

I am not sure why this was not uncovered with "./test-apparmor.py -v" in q-r-t until we run test cases one-by-one now.

[John Johansen (jjohansen)]

Note, Bionic and Focal will likely require more than just that patch. There were multiple patches for Python 3 support landed in apparmor 3. I am not sure what the exact combination required will be.

[Po-Hsu Lin (cypressyew)]

Yes John's comment is correct, we can't just run `make check` for source cloned from launchpad directly. It will complain about:
  aa-genprof:60: undefined name 'PermissionError'
  aa-genprof:130: undefined name 'PermissionError'

I will update the bug description.

qa-regression-testing will have some local patches[1] applied before starting the test. We must use it to get a working version.

To experiment with the patched version via qa-regression-testing, comment out the following lines (1583, 1584) in tearDown()
        if os.path.exists(self.tmpdir):
            testlib.recursive_rm(self.tmpdir)

With the patched apparmor-2.13.3, I can just remove the "python2" string from apparmor-2.13.3/common/Make.rules file and run the following commands without any error:
  export USE_SYSTEM=1
  make check

Please find attachment for the test output.

In this case we might be able to add another local patch in qa-regression-testing for this change.
[1] https://git.launchpad.net/qa-regression-testing/tree/scripts/apparmor/patches

[Po-Hsu Lin (cypressyew)]

Considering the fact that fixing this in apparmor might need many other changes (as presented in the qa-regression-testing/tree/scripts/apparmor/patches directory) I think it's probably easier to add another fix there.

And if we're removing python2, we can remove test_utils_testsuite() and just keep test_utils_testsuite3() which is a test with python3.

This will require some tests with older releases.

[Po-Hsu Lin (cypressyew)]

Note that this issue was not uncovered when running the whole test suite in test-apparmor.py from qa-regression-testing is because of bug 2000233