UFW interferes with LXD Containers network communication

[Sean Feole]

Deploying swift-storage-charm on archs S390X, arm64, amd64 causes UFW to enable and only allow ssh communication to the host & LXD containers. This breaks all heartbeats to/from JUJU and any other application:

The host is running:

$ uname -a
Linux s4lpa 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:07 UTC 2018 s390x s390x s390x GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

prior to deploying swift-storage-charm ufw is listed as inactive on the host:

$ sudo ufw status
Status: inactive

After the swift-storage-charm is deployed , the following occurs.

$ sudo ufw status
Status: active

To Action From
-- ------ ----
873 ALLOW Anywhere
22 ALLOW Anywhere
873 (v6) ALLOW Anywhere (v6)
22 (v6) ALLOW Anywhere (v6)

ubuntu@s4lpa:/var/log$ sudo cat syslog | grep UFW
Feb 1 09:56:01 localhost kernel: [ 2506.078736] [UFW BLOCK] IN=bridge0 OUT=bridge0 MAC=02:00:00:ca:31:39:02:00:00:d4:c1:8a:08:00 SRC=10.13.3.2 DST=10.13.3.219 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=38891 DF PROTO=TCP SPT=17070 DPT=60766 WINDOW=1512 RES=0x00 ACK URGP=0
Feb 1 09:56:01 localhost kernel: [ 2506.202631] [UFW BLOCK] IN=bridge0 OUT=bridge0 MAC=02:00:00:ca:31:39:02:00:00:d4:c1:8a:08:00 SRC=10.13.3.2 DST=10.13.3.234 LEN=115 TOS=0x00 PREC=0x00 TTL=63 ID=2234 DF PROTO=TCP SPT=17070 DPT=53736 WINDOW=1263 RES=0x00 ACK PSH URGP=0
Feb 1 09:56:01 localhost kernel: [ 2506.204782] [UFW BLOCK] IN=bridge0 OUT=bridge0 MAC=02:00:00:ca:31:39:02:00:00:d4:c1:8a:08:00 SRC=10.13.3.2 DST=10.13.3.235 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=38937 DF PROTO=TCP SPT=17070 DPT=46064 WINDOW=1512 RES=0x00 ACK URGP=0
Feb 1 09:56:01 localhost kernel: [ 2506.229545] [UFW BLOCK] IN=bridge0 OUT=bridge0 MAC=02:00:00:ca:31:39:02:00:00:d4:c1:8a:08:00 SRC=10.13.3.2 DST=10.13.3.254 LEN=52 TOS=0x00

Therefore, causing juju to lose access to the containers, and prevent any network access from outside of the host.

:~> juju machines
Machine State DNS Inst id Series AZ Message
0 started 10.13.3.10 manual:10.13.3.10 xenial Manually provisioned machine
0/lxd/0 down 10.13.3.254 juju-b5e612-0-lxd-0 xenial Container started
0/lxd/1 down 10.13.3.210 juju-b5e612-0-lxd-1 xenial Container started
0/lxd/2 down 10.13.3.234 juju-b5e612-0-lxd-2 xenial Container started
0/lxd/3 down 10.13.3.235 juju-b5e612-0-lxd-3 xenial Container started
0/lxd/4 down 10.13.3.219 juju-b5e612-0-lxd-4 xenial Container started

Upon disabling UFW on the host, access to the juju containers is reestablished and normal process resumes as expected.

[David Ames (thedac)]

The root cause is the default policy for FORWARD being set to DROP. Traffic to and from the containers go through the forwarding table. This was causing the containers to fail to communicate with the Juju controller.

This change explicitly sets default policy to ALLOW for both FORWARD and OUTPUT.

https://review.openstack.org/540430

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/540430
Committed: https://git.openstack.org/cgit/openstack/charm-swift-storage/commit/?id=6c549d8018f6d1ec9a83042be4039c7c6466bb18
Submitter: Zuul
Branch: master

commit 6c549d8018f6d1ec9a83042be4039c7c6466bb18
Author: David Ames <email address hidden>
Date: Fri Feb 2 07:57:19 2018 -0800

    Set ufw default policy for routed and outgoing

    The charm neglected to set default policy for routed which cases
    applications deployed to containers to fail to deploy.

    This change explicitly sets default policy allow for routed and
    outgoing.

    Closes-Bug: #1747032

    Change-Id: I440e5e040ad433bf0cf6c4ce99439da456476914