firewall-control interface doesn't give snap access to /run/ufw.lock

[Joseph Borg]

We need to drive some ufw commands with MicroK8s in order to make sure all the required ports are open.

firewall-control is connected, but access to the host's /run/ufw.lock is still blocked by AppArmor:

[ 988.736670] audit: type=1400 audit(1628779784.501:2736): apparmor="DENIED" operation="file_lock" profile="snap.microk8s.daemon-kubelite" name="/run/ufw.lock" pid=46735 comm="python3" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
[ 989.508280] audit: type=1400 audit(1628779785.273:2737): apparmor="DENIED" operation="file_lock" profile="snap.microk8s.daemon-kubelite" name="/run/ufw.lock" pid=46843 comm="python3" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
[ 990.258163] audit: type=1400 audit(1628779786.021:2738): apparmor="DENIED" operation="file_lock" profile="snap.microk8s.daemon-kubelite" name="/run/ufw.lock" pid=46899 comm="python3" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
[ 991.015444] audit: type=1400 audit(1628779786.781:2739): apparmor="DENIED" operation="file_lock" profile="snap.microk8s.daemon-kubelite" name="/run/ufw.lock" pid=46955 comm="python3" requested_mask="k" denied_mask="k" fsuid=0 ouid=0
[ 991.756865] audit: type=1400 audit(1628779787.521:2740): apparmor="DENIED" operation="file_lock" profile="snap.microk8s.daemon-kubelite" name="/run/ufw.lock" pid=47060 comm="python3" requested_mask="k" denied_mask="k" fsuid=0 ouid=0

[Alberto Mardegan (mardy)]

How can I trigger this error? Just running `snap start microk8s` does not seem to raise it.

[Joseph Borg (joeborg)]

Please try then attaching the interfaces specified in https://github.com/ubuntu/microk8s/blob/feature/jdb/strict/docs/build.md#connecting-the-required-interfaces

[Alberto Mardegan (mardy)]

They are already all connected, but maybe I'll remove the snap and try it all again from the beginning.

However, I got some very nice (although not positive!) feedback on my branch: https://github.com/snapcore/snapd/pull/10613#pullrequestreview-729689088

Your input would be valuable, too. I guess that the fastest path forward for now is to add the rule to the kubernetes-support interface, but the point of compatibility between different ufw versions, and possible conflicts with configuration files from the host indeed suggest that using the ufw from the host system might be the better option. Then of course, we would need to add ufw to the core snap.

[Joseph Borg (joeborg)]

Thanks! Not sure whether to put the feedback here, but I'll put some on the PR.

[Joseph Borg (joeborg)]

TLDR, we can't seem to use layouts to layout the /run/ufw.lock to $SNAP_DATA/run/ufw.lock

[Ian Johnson (anonymouse67)]

@joeborg, well given Jamie's comments, I don't think that microk8s should be using it's own separate lock for ufw, I think it should be interacting with ufw on the host system and using that lock otherwise it sounds like things are bound to go wrong with conflicts between the two instances of ufw working at the same time with different lock files.

Can you detail how microk8s uses ufw?

[Joseph Borg (joeborg)]

@anonymouse67 We use it to ensure that the interfaced created my MicroK8s are open: https://github.com/ubuntu/microk8s/blob/0f22722c0d2dcbbebb75a6b7c70a717d54dbd68d/microk8s-resources/wrappers/run-kubelite-with-args#L77

[Ian Johnson (anonymouse67)]

@joeborg given Jamie's comment on the PR, what do you think about making that snippet of code where you run ufw specific to only run when the snap is in classic confinement on classic devices (and then furthermore to make sure to only call ufw from the host and not bundle/ship ufw in your snap)?

The thinking here as I understand it from reading Jamie's response is that ufw is not (yet) made to handle forwards and backwards compatibility, so if you are using ufw from inside your snap, you are most likely using a different version of ufw than exists on the host and are subject to the potential issue where the host's ufw and your snap's ufw disagree and break or worst case somehow disagree and end up opening up the user's firewall entirely.

If you limit your usage of ufw to only when the snap is installed via classic confinement, then you can reach out to normal $PATH and call ufw from the host (thus ensuring that you are using the global lock that matches whatever a normal sysadmin would use if they called ufw themselves outside of confinement and also that the policy generated by ufw from the host matches and is compatible). This also means that by default this code will never run on Ubuntu Core, since classic snaps cannot and never will be installable on Ubuntu Core.

This has the inverse conclusion that if you know your snap is running on Ubuntu Core, you also know that there is not a ufw "from the host", since ufw does not come from any of the base snaps.

Now the issue with the above of course is what if someone is:
1. Has a classic system
2. Is using ufw to manage their firewall
3. Tries to use the strict snap on their device

I don't know whether this is a supported configuration or if it's an acceptable solution to have this user just use the classic snap. If not, you could again have code specific to this situation in microk8s and do as Jamie recommends for this to:

> b) convert its ufw commands to iptables/netfilter commands (setting up microk8s-specific tables so it plays nicely with other software; see ufw, libvirt, lxd, et al (multipass?) for examples)

Finally, you could just sidestep using ufw altogether and just go with Jamie's suggestion b) everywhere.