seccomp missing many new syscalls

[Jamie Strandboge]

[Impact]
Several syscalls were discovered to be missing when using the launcher on snappy. These should be added so we may properly support seccomp filtering.

[Test Case]
seccomp itself has a comprehensive testsuite, and while it doesn't fail the build, regressions can be seen by looking at the build log. Eg:

Regression Test Summary
tests run: 6494
tests skipped: 52
tests passed: 6494
tests failed: 0
tests errored: 0

Furthermore, on a snappy system, perform:
# Note, for the 14.04 SRU, you'll have to install snapd from trusty-proposed and reboot into the lts kernel that it installs
$ sudo snap install hello-world
$ hello-world.env

It should show the environment. On an arm system with 2.1.1-1 from the archive, this will fail due to a seccomp denial:
audit: type=1326 audit(1430766107.122:16): auid=1000 uid=1000 gid=1000 ses=15 pid=1491 comm="env" exe="/bin/bash" sig=31 arch=40000028 syscall=983045 compat=0 ip=0xb6fb0bd6 code=0x0

(note, snappy images have a ppa fix for this, see notes below).

To test the segfault fix, do:
$ scmp_sys_resolver 1024
Segmentation fault

It should return:
$ scmp_sys_resolver 1024
UNKNOWN

For the new 3.19 syscalls:
$ scmp_sys_resolver getrandom
-1

it should return something like (actual number depends on arch, this is on armhf):
$ scmp_sys_resolver getrandom
384

For the 14.04 SRU, test the following syscalls (expected results on amd64 are shown):

$ scmp_sys_resolver getrandom
318
$ scmp_sys_resolver membarrier
324
$ scmp_sys_resolver userfaultfd
323
$ scmp_sys_resolver mlock2
325

autopkgtests for libseccomp have been added as part of this update to verify that the library recognizes all the syscalls from 3.19 and the private syscalls. These tests can be run like so (assuming you are in the unpacked source and the binaries are in ../binary):
$ export REL=vivid
$ adt-run `for i in ../binary/*.deb ; do echo -n "-B $i " ; done` --source ../source/*.dsc --log-file /tmp/adt.out --- adt-virt-schroot autopkgtest-$REL-amd64 || echo "** AUTOPKGTESTS FAILED"

Alternatively, if you don't have autopkgtest setup, you can do:
$ apt-get install dpkg-dev build-essential linux-libc-dev libseccomp-dev seccomp
$ export ADTTMP=/tmp/foo ; mkdir -p "$ADTTMP" ; sh ./debian/tests/test-filter
...
PASS
$ export ADTTMP=/tmp/foo ; mkdir -p "$ADTTMP" ; sh ./debian/tests/test-scmp_sys_resolver
...
PASS

Lastly, seccomp is used by lxc. lxc can be tested by using the test case as outlined in step 4 of https://wiki.ubuntu.com/Process/Merges/TestPlans/AppArmor#Desktop_only.

[Regression Potential]
If the above tests, regression potential is considered low. Unknown syscalls will continue to be handled as before.

Description of changes:
add finit_module:
https://github.com/seccomp/libseccomp/commit/64152018ffdf971efefd84466db4a92002bb8b15

sync the syscall table entries - 3.16
https://github.com/seccomp/libseccomp/commit/9186136be7696ed63a8ddc06c9b397057abc5c75
https://github.com/seccomp/libseccomp/commit/3f319a9a5bc2e32f5a3c296fb0476c040b6f46c4
https://github.com/seccomp/libseccomp/commit/689f19e7488535c775c1db415b8d9895905ef8dd
https://github.com/seccomp/libseccomp/commit/ac6802b300922ef2ad3e95e2c80f89b575073aeb
https://github.com/seccomp/libseccomp/commit/c6205d9600983aa3fa68ca952b7624f2fec86718
https://github.com/seccomp/libseccomp/commit/76739812a3e23182504cde43403ddb9921e0e05a

sync the syscall table entries - 3.17
https://github.com/seccomp/libseccomp/commit/6354f8cab5ac82a8d567005e58a9e7ff9dd843a9

sync the syscall table entries - 3.19
https://github.com/seccomp/libseccomp/commit/7b80fb2fb683cafaf5dc9ff7692437ba86e598a3

This should also be applied (fix a segfault for invalid syscall numbers):
https://github.com/seccomp/libseccomp/commit/2d09a74c7f04d29ae740db1e2187ff1a1886b2c3

For the 14.04 SRU so that libseccomp can handle all of the syscalls in the 4.4 based linux-lts-xenial kernel:
- membarrier and userfaultfd syscalls:
  https://github.com/seccomp/libseccomp/commit/d2ca11b7cdddbba3782b1e306ceacf19e898faee
- x86 direct socket syscalls
  https://github.com/seccomp/libseccomp/commit/24114ca6703036f76be1920a7ba387d6835dd764
- mlock2 syscall
  https://github.com/seccomp/libseccomp/commit/173b96ba8d36a4b1954e99570e82f2f932fe056a

In addition, add-missing-arm-private-syscalls.patch is add to add 5 private ARM syscalls. These are absolutely required on snappy. This portion of the patch has been well tested and is included by default in stable snappy images via the snappy image PPA.

[Jamie Strandboge (jdstrand)]

Uploaded the same version to wily (it is in unapproved).

[Launchpad Janitor (janitor)]

This bug was fixed in the package libseccomp - 2.1.1-1ubuntu1~vivid1

---------------
libseccomp (2.1.1-1ubuntu1~vivid1) vivid-proposed; urgency=medium

  * add-finit-module.patch: add finit_module syscalls to x86 and x86-64
    syscall tables
  * update syscalls for modern kernels (skipping MIPS)
    - update syscalls for 3.16:
      + update-x86-syscall-table.patch
      + update-x86_64-syscall-table.patch
      + update-arm-syscall-table.patch
      + update-x32-syscall-table.patch
      + sync-syscall-table-entries.patch
      + sync-syscall-table-entries-fixtypo.patch
    - update syscalls for 3.17:
      + sync-syscall-table-entries-3.17.patch
    - update syscalls for 3.19:
      + sync-syscall-table-entries-3.19.patch
    - LP: #1450642
  * fix-segfault-with-unknown.patch: fix segfault when find unknown syscall
  * debian/patches/add-missing-arm-private-syscalls.path: add missing private
    ARM syscalls
  * add autopkgtests for scmp_sys_resolver and filter testing and
    SYS_getrandom() testing

 -- Jamie Strandboge <email address hidden> Mon, 04 May 2015 13:53:49 -0500

[Chris J Arges (arges)]

Since 2.1.1-1ubuntu1~vivid1 is already in wily, I cannot accept this into vivid. Can you change the string in wily to not include ~vivid1 please?

[Brian Murray (brian-murray)]

Actually, because that version number, 2.1.1-1ubuntu1~vivid1, has already been used in wily, we can't use the same version number in vivid. I'll reject the upload.

[Jamie Strandboge (jdstrand)]

2.1.1-1ubuntu1~vivid2 uploaded. I'll upload 2.1.1-1ubuntu1 to wily in a few minutes.

[Chris J Arges (arges)]

Hello Jamie, or anyone else affected,

Accepted libseccomp into vivid-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/libseccomp/2.1.1-1ubuntu1~vivid2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Jamie Strandboge (jdstrand)]

Test cases pass with packages in vivid-proposed:
 * in build testsuite results are same between previous build and this for all archs: PASS
 * scmp_sys_resolver 1024: PASS
 * scmp_sys_resolver getrandom: PASS
 * autopkgtests: PASS
 * lxc (amd64 and i386 only): PASS
 * docker framework (snappy/armhf): PASS
 * snappy hello-world.env (snappy/armhf): PASS

As a further data point, this package is source-identical to what is in wily (other than the debian/changelog) and wily's update made it through proposed migration and lxc's tests all passed:
https://jenkins.qa.ubuntu.com/job/wily-adt-lxc/lastBuild/ARCH=amd64,label=adt/artifact/results/log
https://jenkins.qa.ubuntu.com/job/wily-adt-lxc/lastBuild/ARCH=i386,label=adt/artifact/results/log

[Launchpad Janitor (janitor)]

This bug was fixed in the package libseccomp - 2.1.1-1ubuntu1~vivid2

---------------
libseccomp (2.1.1-1ubuntu1~vivid2) vivid-proposed; urgency=medium

  * add-finit-module.patch: add finit_module syscalls to x86 and x86-64
    syscall tables
  * update syscalls for modern kernels (skipping MIPS)
    - update syscalls for 3.16:
      + update-x86-syscall-table.patch
      + update-x86_64-syscall-table.patch
      + update-arm-syscall-table.patch
      + update-x32-syscall-table.patch
      + sync-syscall-table-entries.patch
      + sync-syscall-table-entries-fixtypo.patch
    - update syscalls for 3.17:
      + sync-syscall-table-entries-3.17.patch
    - update syscalls for 3.19:
      + sync-syscall-table-entries-3.19.patch
    - LP: #1450642
  * fix-segfault-with-unknown.patch: fix segfault when find unknown syscall
  * debian/patches/add-missing-arm-private-syscalls.path: add missing private
    ARM syscalls
  * add autopkgtests for scmp_sys_resolver and filter testing and
    SYS_getrandom() testing

 -- Jamie Strandboge <email address hidden> Fri, 08 May 2015 17:10:14 -0400

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for libseccomp has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Steve Langasek (vorlon)]

Hello Jamie, or anyone else affected,

Accepted libseccomp into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libseccomp/2.1.1-1ubuntu1~trusty1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Tyler Hicks (tyhicks)]

I've completed my verification of the libseccomp 2.1.1-1ubuntu1~trusty1 SRU.

I followed the test plan and everything went as expected. I think this SRU is good to go.

[Jamie Strandboge (jdstrand)]

Note bug #1653487 which says that this SRU is not enough to get seccomp working with snaps on 64 bit systems.

[Tyler Hicks (tyhicks)]

I wanted to mention that snaps were working with libseccomp from trusty-proposed in my testing. I tested with the hello-world, pwgen-tyhicks, and lxd snaps on amd64. However, bug #1653487 shows there is a snapd build test failure with the libseccomp from trusty-proposed and it needs to be triaged to understand what's breaking.

[Brian Murray (brian-murray)]

I'm setting the tag back to verification-needed then to prevent this from being released.

[Jamie Strandboge (jdstrand)]

@Tyler, the problem in bug #1653487 was due to a latent bug in libseccomp 2.1 that is only exposed via snap-confine's use of argument filtering. I'm uploading 2.1.1-1ubuntu1~trusty3 now and will do the verification.

[Brian Murray (brian-murray)]

Hello Jamie, or anyone else affected,

Accepted libseccomp into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libseccomp/2.1.1-1ubuntu1~trusty3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Jamie Strandboge (jdstrand)]

I've completed my verification of 2.1.1-1ubuntu1~trusty3 SRU for amd64 and i386.

I followed the test plan for this and bug #1653487 with additional manual testing for lxc and docker debs along with various snaps (ufw, lxd, docker (amd64 only since docker upstream doesn't provide 32 bit images; also see unrelated bug #1654590), etc and found no regressions.

NOTE: I discovered that the systemd upstart job won't start if cgmanager (pulled in by lxc) is installed. tvoss and I discussed and he will followup with Foundations. This is unrelated to this SRU but may be useful to know for testers.

[Robie Basak (racb)]

There's an autopkgtest lxc failure on armhf, which usually passes. Please could you determine if this is caused by this SRU? Removing verification-done for now to avoid any accidents. Please set back if you determine that the autopkgtest failure has a cause unrelated to this SRU.

[Robie Basak (racb)]

http://autopkgtest.ubuntu.com/packages/l/lxc/trusty/armhf

[Jamie Strandboge (jdstrand)]

There are a lot of failures and containers don't seem to be starting for a variety of reasons. lxc 1.0.8 (what this version of libseccomp was tested against) always failed on armhf according to http://autopkgtest.ubuntu.com/packages/l/lxc/trusty/armhf. 1.0.7 also always failed. 1.0.9 started to pass occasionally, but not reliably.

Case in point: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-trusty/trusty/armhf/l/lxc/20161205_235522_59b84@/log.gz uses the libseccomp from the release pocket and it has:

$ zgrep -i fail ./log.gz
FAIL: lxc-tests: /usr/bin/lxc-test-attach
FAIL: lxc-tests: /usr/bin/lxc-test-autostart
FAIL
FAIL: lxc-tests: /usr/bin/lxc-test-cgpath
FAIL: lxc-tests: /usr/bin/lxc-test-concurrent
Starting the container (lxc-test-concurrent-2) failed...
Starting the container (lxc-test-concurrent-4) failed...
Starting the container (lxc-test-concurrent-1) failed...
Starting the container (lxc-test-concurrent-0) failed...
FAIL: lxc-tests: /usr/bin/lxc-test-console
FAIL: lxc-tests: /usr/bin/lxc-test-createtest
67: failed to start lxctest1
FAIL: lxc-tests: /usr/bin/lxc-test-destroytest
FAIL: lxc-tests: /usr/bin/lxc-test-shutdowntest
68: failed to start lxctest1
FAIL: lxc-tests: /usr/bin/lxc-test-startone
169: lxctest1 failed to start
FAIL: lxc-tests: /usr/bin/lxc-test-symlink
lxc-start: lxc_start.c: main: 341 The container failed to start.
+ pass=fail
+ '[' fail = pass ']'
+ pass=fail
+ '[' fail '!=' pass ']'
      lxc-start 1483662018.638 ERROR lxc_cgfs - cgfs.c:cgfs_init:2246 - cgroupfs failed to detect cgroup metadata
      lxc-start 1483662018.638 ERROR lxc_start - start.c:lxc_spawn:884 - failed initializing cgroup support
      lxc-start 1483662018.711 ERROR lxc_start - start.c:__lxc_start:1121 - failed to spawn 'symtest1'
      lxc-start 1483662018.711 WARN lxc_commands - commands.c:lxc_cmd_rsp_recv:172 - command get_state failed to receive response
      lxc-start 1483662018.712 ERROR lxc_start_ui - lxc_start.c:main:341 - The container failed to start.
+ echo 'FAIL: Test 1: expected pass but container did not.'
FAIL: Test 1: expected pass but container did not.
FAIL: lxc-tests: /usr/bin/lxc-test-ubuntu
lxc_container: lxccontainer.c: create_run_template: 1084 container creation template for lxc-test-ubuntu failed
Failed creating ubuntu container
FAIL: python3: API
SUMMARY: pass=9, fail=12, ignored=5

which are all the same failures as in https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-trusty/trusty/armhf/l/lxc/20170106_002110_b403a@/log.gz (the one with this SRU's libseccomp).

In other words, this SRU introduced no new regressions in the lxc autopkgtest.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libseccomp - 2.1.1-1ubuntu1~trusty3

---------------
libseccomp (2.1.1-1ubuntu1~trusty3) trusty-proposed; urgency=medium

  * Cherrypick various bpf fixes to support argument filtering on 64-bit
    (LP: #1653487)
    - debian/patches/bpf-use-state-arch.patch: use state->arch instead of
      db->arch in _gen_bpf_arch()
    - debian/patches/db-require-filters-to-share-endianess.patch: require all
      filters in a collection to share the same endianess
    - debian/patches/resolve-issues-caused-by-be.patch: resolve issues caused
      by big endian systems
    - debian/patches/bpf-accumulator-check.patch: test the bpf accumulator
      checking logic
    - debian/patches/bpf-track-accumulator-state.patch: track accumulator
      state and reload it when necessary. This is the fix for LP: #1653487. The
      previous patches are required by this patch.
    - debian/patches/ensure-simulator-has-valid-arch.patch: ensure the
      simulator always has a valid architecture value. This fixes a regression
      in the testsuite introduced by resolve-issues-caused-by-be.patch
    - debian/patches/bpf-accumulator-check-indep.patch: fix a regression in the
      testsuite introduced by bpf-accumulator-check.patch
    - debian/patches/fix-audit-arch-i386.patch: fix arch token for 32-bit x86
      not being defined correctly for the tools

libseccomp (2.1.1-1ubuntu1~trusty1) trusty-proposed; urgency=medium

  * Bring libseccomp 2.1.1-1ubuntu1~vivid2, from Ubuntu 14.10, to Ubuntu
    14.04 and add a couple patches to account for new syscalls found in the
    4.4 based hardware enablement kernel. This allows for proper snap seccomp
    confinement on Ubuntu 14.04 when using the hardware enablement kernel
    (LP: #1450642)
    - debian/patches/add-membarrier-and-userfaultfd.patch: Add membarrier and
      userfaultfd syscalls
    - debian/patches/add-mlock2.patch: Add mlock2 syscall
    - debian/tests/data/all-except-s390-4.4.filter: Add autopkgtest that
      verifies all syscalls found in the 4.4 kernel, except for the s390
      specific syscalls, are supported by libseccomp. The s390 specific
      syscalls are not needed since this version of libseccomp does not
      support the s390 architecture.
    - debian/tests/test-filter: Skip the getrandom filter tests since
      SYS_getrandom is not defined in 14.04 environment and the getrandom(2)
      syscall is not even available in the 14.04 release kernel.

libseccomp (2.1.1-1ubuntu1~vivid2) vivid-proposed; urgency=medium

  * add-finit-module.patch: add finit_module syscalls to x86 and x86-64
    syscall tables
  * update syscalls for modern kernels (skipping MIPS)
    - update syscalls for 3.16:
      + update-x86-syscall-table.patch
      + update-x86_64-syscall-table.patch
      + update-arm-syscall-table.patch
      + update-x32-syscall-table.patch
      + sync-syscall-table-entries.patch
      + sync-syscall-table-entries-fixtypo.patch
    - update syscalls for 3.17:
      + sync-syscall-table-entries-3.17.patch
    - update syscalls for 3.19:
      + sync-syscall-table-entries-3.19.patch
    - LP: #1450642
  * fix-segfault-with-unknown.patch: fix segfault when...