The proper documented way of extending/customizing the docker daemon options through systemd is to use the EnvironmentFile directive:
https://docs.docker.com/engine/admin/systemd/#/custom-docker-daemon-options
Right now we pretty much hardcode the docker daemon options without leaving room for additional end-user customization: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/baremetal/templates/docker_systemd_service.j2
An example use case of being more flexible in customization is this particular Docker issue: https://github.com/docker/docker/issues/22297
TL;DR: there can be a mismatch of MTU configuration between the host and the container -- say the host has a MTU of 1400, Docker defaults to 1500 anyway and this causes certain network transactions to fail.
Docker has a way to customize the MTU through the systemd unit file, as such:
ExecStart=/usr/bin/dockerd --mtu 1200
But I have no way to "hook" into that with the current way things work.
The way I would probably see this is when we set up the defaults file, here:
https://github.com/openstack/kolla-ansible/blob/49c0ca3a9cc5ba3874aeb7b5fd2215750a1e25bc/ansible/roles/baremetal/tasks/post-install.yml#L25-L30
First, drop that in /etc/default/docker for Ubuntu or /etc/sysconfig/docker for CentOS and then provide a variable to extend DOCKER_OPTS if need be.
So in the defaults template here: https://github.com/openstack/kolla-ansible/blob/7b5e5591fe7108117221eacfffe6e7edbd33ddfa/ansible/roles/baremetal/templates/docker_defaults.j2
Instead of doing: DOCKER_OPTS=""
I would do: DOCKER_OPTS="{{ custom_docker_opts }}".
This would allow someone to pass an extra variable:
kolla-ansible <...> -e "custom_docker_opts=--mtu 1200"
@David Moreau Simard mind to push patch?