snap-confine profile uses 'include' instead of '#include' which breaks apparmor-utils python tools

Bug #1734038 reported by Po-Hsu Lin on 2017-11-23
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
snapd (Ubuntu)
Undecided
Michael Vogt
Trusty
Undecided
Unassigned
Xenial
Undecided
Unassigned
Zesty
Undecided
Unassigned
Artful
Undecided
Unassigned
Bionic
Undecided
Michael Vogt

Bug Description

Issue found with Xenial kernel 4.4.0-102 and Zesty kernel 4.10.0-41, across different architectures

Multiple tests from ubuntu_qrt_apparmor test suite failed with the same error message:
    ERROR: Syntax Error: Unknown line found in file /etc/apparmor.d/usr.lib.snapd.snap-confine.real line 15:
    include "/var/lib/snapd/apparmor/snap-confine.d" /etc/ld.so.cache r,

(BTW the include and this ld.so.cache are not in the same line, please refer to comment #3 for attachment)

This issue will gone if you downgrade the snapd and ubuntu-core-launcher package:
    sudo apt-get install snapd=2.28.5 ubuntu-core-launcher=2.28.5

Debug information:
ubuntu@kernel01:~$ snap version
snap 2.29.3
snapd 2.29.3
series 16
ubuntu 16.04
kernel 4.4.0-102-generic

ubuntu@kernel01:~$ apt list snapd
Listing... Done
snapd/xenial-proposed,now 2.29.3 s390x [installed]
N: There are 2 additional versions. Please use the '-a' switch to see them.

ubuntu@kernel01:~$ apt list apparmor -a
Listing... Done
apparmor/xenial-updates,now 2.10.95-0ubuntu2.7 s390x [installed]
apparmor/xenial-security 2.10.95-0ubuntu2.6 s390x
apparmor/xenial 2.10.95-0ubuntu2 s390x

Steps to run the Apparmor test from QA Regression testing suite:
  1. git clone --depth 1 https://git.launchpad.net/qa-regression-testing
  2. sudo ./qa-regression-testing/scripts/test-apparmor.py

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-102-generic 4.4.0-102.125
ProcVersionSignature: Ubuntu 4.4.0-102.125-generic 4.4.98
Uname: Linux 4.4.0-102-generic s390x
NonfreeKernelModules: zfs zunicode zcommon znvpair zavl
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.13
Architecture: s390x
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
CurrentDmesg:

Date: Thu Nov 23 01:36:31 2017
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lspci:

Lsusb: Error: command ['lsusb'] failed with exit code 1:
PciMultimedia:

ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=C
 SHELL=/bin/bash
ProcFB: Error: [Errno 2] No such file or directory: '/proc/fb'
ProcKernelCmdLine: root=UUID=44b0b919-a1a4-4849-9425-e71d4ac87d85 crashkernel=196M BOOT_IMAGE=0
RelatedPackageVersions:
 linux-restricted-modules-4.4.0-102-generic N/A
 linux-backports-modules-4.4.0-102-generic N/A
 linux-firmware 1.157.13
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)

Po-Hsu Lin (cypressyew) wrote :

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1734038

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
Po-Hsu Lin (cypressyew) on 2017-11-23
description: updated
tags: added: regression-proposed

The /etc/apparmor.d/usr.lib.snapd.snap-confine.real file from affected system.

Michael Vogt (mvo) wrote :

We believe this is a kernel/apparmor issue. The apparmor documentation (http://wiki.apparmor.net/index.php/AppArmor_Core_Policy_Reference#Include_statements) lists "include" as a valid include. We are happy to fix snapd but I'm sure there a many existing profiles (beside snapd from other packages or user written ones) that will break if this change lands.

Po-Hsu Lin (cypressyew) on 2017-11-23
description: updated
Po-Hsu Lin (cypressyew) wrote :

Tested with snapd 2.29.3+17.10 on Artful with kernel 4.13.0-17 (the proposed one will be 4.13.0-18), this issue can be reproduced.

Michael Vogt (mvo) wrote :

Could you please also include the output of "snap version" and "apt list snapd" ?

Po-Hsu Lin (cypressyew) on 2017-11-23
description: updated
description: updated
Po-Hsu Lin (cypressyew) on 2017-11-23
description: updated
Andy Whitcroft (apw) wrote :

Ok this seems to be an issue with some of the basic apparmor commands not preprocessing the profiles when working on them.

If we ask apparmor to parse the file in question it is happy to do so:

apparmor_parser -p -Q /etc/apparmor.d/snap.core.3440.usr.lib.snapd.snap-confine
[...]
    # Those are discussed on https://forum.snapcraft.io/t/snapd-vs-upstream-kernel-vs-apparmor
    # and https://forum.snapcraft.io/t/snaps-and-nfs-home/

##included "/var/lib/snapd/apparmor/snap-confine.d"

    # We run privileged, so be fanatical about what we include and don't use
    # any abstractions
    /etc/ld.so.cache r,
[...]

However, it does not seem to handle this well when we use some of the associated utilities:

$ sudo aa-complain foo

ERROR: Syntax Error: Unknown line found in file /etc/apparmor.d/snap.core.3440.usr.lib.snapd.snap-confine line 15:
    include "/var/lib/snapd/apparmor/snap-confine.d" /etc/ld.so.cache r,

$ sudo aa-disable foo

ERROR: Syntax Error: Unknown line found in file /etc/apparmor.d/snap.core.3440.usr.lib.snapd.snap-confine line 15:
    include "/var/lib/snapd/apparmor/snap-confine.d" /etc/ld.so.cache r,

Andy Whitcroft (apw) wrote :

These errors are throwing the qrt-apparmor test suite out.

Andy Whitcroft (apw) wrote :

This appears to be being triggered because there are actually two independent profile parsers in apparmor. There is a C version used by the apparmor_parse which correctly interprets the 'include "xxx"' syntax and then loads the profile. There is also a python parser (in aa.py) which only seems to understand the 'include <xxx>' syntax and it is this which throws errors when running the utility commands.

Changed in snapd (Ubuntu):
status: New → Invalid
Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Christian Boltz (cboltz) wrote :

> There is also a python parser (in aa.py) which only seems to understand the 'include <xxx>'
> syntax and it is this which throws errors when running the utility commands.

Exactly, that's the cause of this bug. I'll change the title to make it obvious.

Interestingly, it has been this way for years (I checked 2.9, but it probably also affects even older versions) without someone noticing it. Therefore this bug doesn't qualify as regression IMHO ;-)

summary: - Potential regression found with apparmor test on Xenial/Zesty
+ utils don't understand «include "/where/ever"» (was: Potential
+ regression found with apparmor test on Xenial/Zesty)

Yes, the split parser has been a issue for a long time. There has been a plan to make the flex/yacc/C parser code available as a lib for the other tools but its one of those things that never gets resources allocated.

The short term fix for this is probably a backport of a newer version of the python utils.

Jamie Strandboge (jdstrand) wrote :

@mvo - this is probably obvious, but if you used '#include' instead of 'include', it would side-step the issue.

Michael Vogt (mvo) wrote :
Jamie Strandboge (jdstrand) wrote :

Since snapd is using this bug for its SRU blocker and we have bug #1733700 that is the same issue, I'm going to use this bug as the snapd one and for the apparmor one.

summary: - utils don't understand «include "/where/ever"» (was: Potential
- regression found with apparmor test on Xenial/Zesty)
+ snap-confine profile uses 'include' instead of '#include' which breaks
+ apparmor-utils python toolsnd with apparmor test on Xenial/Zesty)
no longer affects: apparmor
summary: snap-confine profile uses 'include' instead of '#include' which breaks
- apparmor-utils python toolsnd with apparmor test on Xenial/Zesty)
+ apparmor-utils python tools
no longer affects: apparmor (Ubuntu)
no longer affects: linux (Ubuntu)
Changed in snapd (Ubuntu):
assignee: nobody → Michael Vogt (mvo)
status: Invalid → In Progress
Jamie Strandboge (jdstrand) wrote :

FYI, we discussed how to move forward on this:

* mvo will update the SRU deb in -proposed for all releases to include the fix
* mvo will create a 2.29.4.2 stable core snap
* jdstrand will perform an apparmor SRU to fix the tools (tracked in bug #1733700)

@Po-Hsu Lin and @apw - when the deb is updated, qrt should stop failing since it will have a fixed rule

When the deb and stable core snap are available, all non-snap users and all not-yet-snap users will no longer be affected by this bug.

People who have installed snaps and have a core snap >= 2.29 and < 2.29.4.2 installed will still be affected by this bug even when the deb and updated stable core snap become available until the apparmor userspace (bug #1733700) is fixed. This is because snapd keeps up to 3 revisions of snaps on the disk to support rollbacks and the earlier affected snap-confine profiles are still on disk. Until bug #1733700 is fixed, you can look in /snap/core to see what revisions are on disk, then use 'sudo snap remove core --revision=<affected revision>' to remove the older affected revisions and workaround the problem.

My Core is 16-2-.29.4.2 and I am seeing the issue as
sudo aa-complain foo

ERROR: Syntax Error: Unknown line found in file /etc/apparmor.d/snap.core.3440.usr.lib.snapd.snap-confine line 15:
    include "/var/lib/snapd/apparmor/snap-confine.d" /etc/ld.so.cache r,

Changed in snapd (Ubuntu Xenial):
status: New → Fix Released
Changed in snapd (Ubuntu Zesty):
status: New → Invalid
Jamie Strandboge (jdstrand) wrote :

@Matthew, your bug is now being tracked separately, here: https://bugs.launchpad.net/bugs/1733700

Jamie Strandboge (jdstrand) wrote :

2.29.4 debs and 16-2.30 core snaps use:

  #include "/var/lib/snapd/apparmor/snap-confine.d"

This is sufficient to mark this bug Fix Released for snapd. For systems that are affected by this bug with older revisions of the core snap, please see the workaround in https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1734038/comments/15 and follow the progress in the apparmor bug: https://bugs.launchpad.net/bugs/1733700

Changed in snapd (Ubuntu Trusty):
status: New → Fix Released
Changed in snapd (Ubuntu Zesty):
status: Invalid → Fix Released
Changed in snapd (Ubuntu Artful):
status: New → Fix Released
Changed in snapd (Ubuntu Bionic):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers