ubuntu_qrt_apparmor: i18n test fails on arm64 Hirsute / Impish

[Paolo Pisati]

It only affects arm64:

...
 06/15 13:09:11 ERROR| utils:0153| [stderr] running i18n

 06/15 13:09:11 ERROR| utils:0153| [stderr] Error: open failed. Test
 'i18n (194) OPEN (octal) "/tmp/sdtest.411763-18905-et6mgO/file_Š_post"
 RW' was expected to 'pass'. Reason for failure 'FAIL: open /tmp/sdtest.411763-18905-et6mgO/file_Š_post failed - Permission denied'

06/15 13:09:11 ERROR| utils:0153| [stderr] Error: open failed. Test 'i18n (195) OPEN (octal) "/tmp/sdtest.411763-18905-et6mgO/file_Ê_post" RW' was expected to 'pass'. Reason for failure 'FAIL: open /tmp/sdtest.411763-18905-et6mgO/file_Ê_post failed - Permission denied'

06/15 13:09:11 ERROR| utils:0153| [stderr] Error: open failed. Test 'i18n (196) OPEN (octal) "/tmp/sdtest.411763-18905-et6mgO/file_ÄŠ_post" RW' was expected to 'pass'. Reason for failure 'FAIL: open /tmp/sdtest.411763-18905-et6mgO/file_ÄŠ_post failed - Permission denied'

06/15 13:09:11 ERROR| utils:0153| [stderr] Error: open failed. Test 'i18n (197) OPEN (octal) "/tmp/sdtest.411763-18905-et6mgO/file_ÅŠ_post" RW' was expected to 'pass'. Reason for failure 'FAIL: open /tmp/sdtest.411763-18905-et6mgO/file_ÅŠ_post failed - Permission denied'
...

Full log without LP manging the formatting: http://paste.ubuntu.com/p/3PHcCWb9Jw/plain/

[Paolo Pisati (p-pisati)]

Full log attached too.

[Paolo Pisati (p-pisati)]

And i was able to reproduce the same issue on Hirsute arm64 with a 5.11 kernel, see the attached log.

[John Johansen (jjohansen)]

The i18n test passing using the upstream version.

[John Johansen (jjohansen)]

The i18n test is also passing when pulled from the apparmor 3.0 source in Impish. So this is a probably something in environment/setup.

[John Johansen (jjohansen)]

So the test-apparmor.py is passing the i18n test in a fressh impish amd64 VM if launched manually.

[John Johansen (jjohansen)]

So far I have not been able to replicate this failure.

[John Johansen (jjohansen)]

I should note that, failure to replicate so far is entirely on x86/amd64. My arm64 tests haven't completed yet.

[Paolo Pisati (p-pisati)]

Thanks for looking into this: indeed, we hit this issue on arm64 only.

[Georgia Garcia (georgiag)]

I was able to reproduce this issue on impish and hirsute arm64.
It seems to be an issue with the versions of bash on impish (5.1.8(1)-release) and hirsute (5.1.4(1)-release).

The bash sub-shell on arm, for UTF-8, is adding an artifact when creating the file.
The test creates a file with the wrong character - due to the bash bug, but the policy is filled with the octal representation of a different character. Since they don't match, we get the error "Permission denied".

The attached patch should work. It sets the default locale to disable UTF-8 multibyte match.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "0001-tests-fix-i18n.sh-regression-test-on-arm64.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Thomas Ward (teward)]

Has the fidelity of the patch attached here been vetted by anyone?

(Seth Arnold asked me to take a look @ this while I'm pushing hard for 1934005 to fix some major chaos so this gets in with that update / SRU)

[Thomas Ward (teward)]

Seth Arnold has affirmed the patch is going to fix things, so it's been reviewed by Seth.

[Steve Beattie (sbeattie)]

Georgia's patch was committed in the upstream apparmor project in https://gitlab.com/apparmor/apparmor/-/commit/458a981b6242e8b1cce1599ca95d89dcd10f60e7 in https://gitlab.com/apparmor/apparmor/-/merge_requests/765 and was cherrypicked to the apparmor-3.0 branch amongst others in https://gitlab.com/apparmor/apparmor/-/commit/0729b13293d775dfd6ec4c9a3b9313b6e125d540 .

[Thomas Ward (teward)]

Thanks for the confirm, Steve. However, these may be 'upstream' but aren't yet in Ubuntu, so. The two issues on my radar are the one I discovered needs fixed for something to properly WORK for torbrowser-launcher, and the other is this one Seth poked at me to include on any SRU for HIrsute. (Or uploads for Impish, they're incoming)

[Steve Beattie (sbeattie)]

The root issue is likely something in the utf-8 handling code in glibc on arm64 hirsute and impish; the reproducer is:

  bash -c 'i=210; echo -n $(printf "\\$(printf "%03o" $i)") | od -An -t uC'
  210 138

running valgrind in a default environemnt (so LANG=en_US.UTF-8) turned up

==46656== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==46653== Conditional jump or move depends on uninitialised value(s)
==46653== at 0x48EA5EC: utf8_internal_loop (loop.c:336)
==46653== by 0x48EA5EC: __gconv_transform_utf8_internal (skeleton.c:620)
==46653== by 0x494BF57: mbrtowc (mbrtowc.c:86)
==46653== by 0x17FFBB: command_substitute (in /usr/bin/bash)

and running the test with LANG=C results in the trailing artifact (138) to not show up.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 3.0.0-0ubuntu9

---------------
apparmor (3.0.0-0ubuntu9) impish; urgency=medium

  * Make X11 socket writable again (LP: #1934005):
    - d/p/ubuntu/lp1934005.patch

  * Fix i18n.sh regression test on arm64 (LP: #1932331)
    - d/p/ubuntu/lp1932331.patch

 -- Thomas Ward <email address hidden> Wed, 30 Jun 2021 17:31:12 -0400

[Brian Murray (brian-murray)]

Hello Paolo, or anyone else affected,

Accepted apparmor into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apparmor/3.0.0-0ubuntu7.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Georgia Garcia (georgiag)]

After downloading the apparmor source from hirsute-proposed and running the regression tests, I was able to confirm that the i18n test is now passing for arm64.

[Launchpad Janitor (janitor)]

This bug was fixed in the package apparmor - 3.0.0-0ubuntu7.1

---------------
apparmor (3.0.0-0ubuntu7.1) hirsute; urgency=medium

  * Make X11 socket writable again (LP: #1934005):
    - d/p/ubuntu/lp1934005.patch

  * Fix i18n.sh regression test on arm64 (LP: #1932331):
    - d/p/ubuntu/lp1932331.patc
    Thanks to Georgia Garcia for the patch.

 -- Thomas Ward <email address hidden> Wed, 30 Jun 2021 17:42:41 -0400

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for apparmor has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.