Comment 0 for bug 1781856

Sometimes you may wish to use a different interface as your default gateway, this is not currently possible for LXD containers. Secondly to that, the default gateway for the "default space" is not used - it appears to use the first space in the database or some other arbitrary ordering.

Currently, with the MAAS provider, while not in the Web UI you can specify a different interface as the default gateway interface using the following command:
$ maas skc interface set-default-gateway wftkqx 914

You specify the machine ID (wftkqx) and the ID of the link that you want to be the default gateway (914). For testing purposes, the following command makes it easy to identify the link IDs for a given machine's interfaces:
maas skc nodes read | jq '.[] | . as $parent | .interface_set[] | "system_id=\($parent.system_id) hostname=\($parent.hostname) if_id=\(.id), if_name=\(.name), if_fabric=\(.vlan.fabric), if_space=\(.vlan.space)"'

Unfortunately this logic does not extend to LXD containers deployed by juju. juju appears to always use the "first" space as the default gateway, and by first I am not sure but I am guessing it is based on the order it appears in the database or similar.

In my setup I have spaces "vsw0" and "vsw1" (in that order in the MAAS interface and "juju spaces" output). When deploying a container with both spaces on a host for which MAAS has the default gateway set to vsw1, it appears vsw0 is always chosen as the container gateway. This includes when you set a different "default space"

juju deploy percona-cluster --bind "vsw1 shared-db=vsw0"

The main reason this cause real problems, is that currently containers don't attempt to do any kind of source routing. So when a container is contacted on one interface, the default gateway for another interface is used. This leads to broken networking in some setups.

This could potentially also be solved by Bug #1737428 which allows for different default routes basic on the traffic "Source IP" - however it makes sense without that support to simply be able to specify the default route for the container - at the very least, perhaps by using the default space or otherwise if that is a problem for some reason perhaps through some other option.