[ubuntu 16.04, lxd 2.0.8 or 2.0.9, tcpdump 4.7.4 or 4.9.0]

If you ssh into an lxd container as a normal user, and inside that container run "sudo tcpdump", the tcpdump process is blocked from writing to stdout/stderr.  This appears to be due to apparmor:  disabling apparmor for tcpdump makes the problem go away.

    ln -s /etc/apparmor.d/usr.sbin.tcpdump /etc/apparmor.d/disable/

Note: this is a different bug from #1641236. In that bug, the user did "lxc exec <container> bash" to get a shell in the container; the stdout fd was being passed from the outer host to the container.  But in this case, the pty is being created entirely inside the container by sshd.

Details copied from https://github.com/lxc/lxd/issues/2930

# Steps to reproduce

1. Create two Ubuntu 16.04 lxd containers, one privileged, one not.
2. ssh into each one, and then use `sudo -s` to get root. (Do not use `lxc exec` because of issue #1641236)
3. Inside one run `tcpdump -i eth0 -nn not tcp port 22`, and ping from the other.

tcpdump in the privileged container works just fine.

tcpdump in the unprivileged container does not show any output. But if I run strace on it I see errors attempting to access stdout and stderr:

~~~
ioctl(1, TCGETS, 0x7fff97c8d680)        = -1 ENOTTY (Inappropriate ioctl for device)
...
write(2, "tcpdump: verbose output suppress"..., 75) = -1 EACCES (Permission denied)
write(2, "listening on eth0, link-type EN1"..., 74) = -1 EACCES (Permission denied)
~~~

This is very weird.  Even more weird: the following command *does* capture packets:

~~~
tcpdump -i eth0 -nn -w foo.pcap
~~~

The file foo.pcap grows. This proves it's nothing to do with network capture perms.

But the following command shows no output:

~~~
tcpdump -r foo.pcap -nn
~~~

And again it's because it can't write to stdout:

~~~
fstat(1, 0x7ffe2fb5eb10)                = -1 EACCES (Permission denied)
read(3, "", 4096)                       = 0
write(1, "14:34:30.618180 IP6 fe80::c609:6"..., 1740) = -1 EACCES (Permission denied)
~~~

I had originally thought this was to do with capabilities.  But if I run `capsh --print` inside both containers, they both have `cap_net_raw` and `cap_net_admin`.  In fact, the unprivileged container has two additional capabilities!  (`cap_mac_override` and `cap_mac_admin`)

So now I suspect that apparmor is at fault.

#### dmesg

dmesg output generated by the following steps:

* start tcpdump
* wait 5 seconds
* send 1 ping from other side
* wait 5 seconds
* stop tcpdump

~~~
[429020.685987] audit: type=1400 audit(1487774339.708:3597): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="/dev/pts/0" pid=12539 comm="tcpdump" requested_mask="wr" denied_mask="wr" fsuid=100000 ouid=101001
[429020.686000] audit: type=1400 audit(1487774339.708:3598): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="/dev/pts/0" pid=12539 comm="tcpdump" requested_mask="wr" denied_mask="wr" fsuid=100000 ouid=101001
[429020.686013] audit: type=1400 audit(1487774339.708:3599): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="/dev/pts/0" pid=12539 comm="tcpdump" requested_mask="wr" denied_mask="wr" fsuid=100000 ouid=101001
[429020.686022] audit: type=1400 audit(1487774339.708:3600): apparmor="DENIED" operation="file_inherit" namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="/dev/pts/0" pid=12539 comm="tcpdump" requested_mask="wr" denied_mask="wr" fsuid=100000 ouid=101001
[429020.716725] device eth0 entered promiscuous mode
[429020.741308] audit: type=1400 audit(1487774339.764:3601): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429020.741330] audit: type=1400 audit(1487774339.764:3602): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429021.716785] audit: type=1400 audit(1487774340.740:3603): apparmor="DENIED" operation="getattr" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="r" denied_mask="r" fsuid=100000 ouid=0
[429030.630448] audit: type=1400 audit(1487774349.652:3604): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630543] audit: type=1400 audit(1487774349.652:3605): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630555] audit: type=1400 audit(1487774349.652:3606): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630565] audit: type=1400 audit(1487774349.652:3607): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630574] audit: type=1400 audit(1487774349.652:3608): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630584] audit: type=1400 audit(1487774349.652:3609): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630593] audit: type=1400 audit(1487774349.652:3610): apparmor="DENIED" operation="file_perm" info="Failed name lookup - disconnected path" error=-13 namespace="root//lxd-srv2-campus1_<var-lib-lxd>" profile="/usr/sbin/tcpdump" name="apparmor/.null" pid=12539 comm="tcpdump" requested_mask="w" denied_mask="w" fsuid=100000 ouid=0
[429030.630687] device eth0 left promiscuous mode
~~~

#### privileged container

~~~
$ lxc config show srv1-campus1
name: srv1-campus1
profiles:
- br-cnd11
config:
  security.privileged: "true"
  volatile.base_image: 315bedd32580c3fb79fd2003746245b9fe6a8863fc9dd990c3a2dc90f4930039
  volatile.eth0.hwaddr: 00:16:3e:25:c5:bf
  volatile.idmap.base: "0"
  volatile.idmap.next: '[]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
devices:
  root:
    path: /
    type: disk
ephemeral: false

root@srv1:~# capsh --print
Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_syslog,cap_wake_alarm,cap_block_suspend,37+ep
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_syslog,cap_wake_alarm,cap_block_suspend,37
Securebits: 00/0x0/1'b0
 secure-noroot: no (unlocked)
 secure-no-suid-fixup: no (unlocked)
 secure-keep-caps: no (unlocked)
uid=0(root)
gid=0(root)
groups=0(root)
~~~

#### unprivileged container

~~~
$ lxc config show srv2-campus1
name: srv2-campus1
profiles:
- br-cnd11
config:
  volatile.base_image: 315bedd32580c3fb79fd2003746245b9fe6a8863fc9dd990c3a2dc90f4930039
  volatile.eth0.hwaddr: 00:16:3e:33:9c:29
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  root:
    path: /
    type: disk
ephemeral: false

root@srv2:~# capsh --print
Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,37+ep
Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,37
Securebits: 00/0x0/1'b0
 secure-noroot: no (unlocked)
 secure-no-suid-fixup: no (unlocked)
 secure-keep-caps: no (unlocked)
uid=0(root)
gid=0(root)
groups=0(root)
~~~