During a kolla-ansible deploy, I frequently have encountered the following failure:
fatal: [localhost]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/tmp/ansible_kolla_docker_payload_OeUe5x/__main__.py\", line 835, in main\\n result = bool(getattr(dw, module.params.get(\\'action\\'))())\\n File \"/tmp/ansible_kolla_docker_payload_OeUe5x/__main__.py\", line 712, in create_volume\\n if not self.check_volume():\\n File \"/tmp/ansible_kolla_docker_payload_OeUe5x/__main__.py\", line 275, in check_volume\\n for vol in self.dc.volumes()[\\'Volumes\\'] or list():\\n File \"/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py\", line 34, in wrapper\\n return f(self, *args, **kwargs)\\n File \"/usr/local/lib/python2.7/dist-packages/docker/api/volume.py\", line 38, in volumes\\n return self._result(self._get(url, params=params), True)\\n File \"/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py\", line 46, in inner\\n return f(self, *args, **kwargs)\\n File \"/usr/local/lib/python2.7/dist-packages/docker/api/client.py\", line 191, in _get\\n return self.get(url, **self._set_request_timeout(kwargs))\\n File \"/usr/local/lib/python2.7/dist-packages/requests/sessions.py\", line 521, in get\\n return self.request(\\'GET\\', url, **kwargs)\\n File \"/usr/local/lib/python2.7/dist-packages/requests/sessions.py\", line 508, in request\\n resp = self.send(prep, **send_kwargs)\\n File \"/usr/local/lib/python2.7/dist-packages/requests/sessions.py\", line 618, in send\\n r = adapter.send(request, **kwargs)\\n File \"/usr/local/lib/python2.7/dist-packages/requests/adapters.py\", line 521, in send\\n raise ReadTimeout(e, request=request)\\nReadTimeout: UnixHTTPConnectionPool(host=\\'localhost\\', port=None): Read timed out. (read timeout=60)\\n'"}
The ReadTimeout occurs on different tasks. The underlying cause is that the docker daemon is not responding within the default 60 second timeout. The workaround when running docker or docker-compose is to set timeout environment variables, e.g.:
export DOCKER_CLIENT_TIMEOUT=120
export COMPOSE_HTTP_TIMEOUT=120
See also https://github.com/docker/compose/issues/3927
However, I have been unable to set these directly in the kolla-ansible playbook 'ansible/site.yml' (via environment variables) and setting the environment variables in bash has no affect on the read timeout.
Does anyone know how to set the docker timeout variable variables in kolla-ansible?
I have temporarily set the nice priority value of the dockerd process to -10 which does avoid the the issue some of the time. Setting the timeout value would be the correct solution.
stable/queens release
I don't think this is currently possible. From looking at the docker-py code [1], we should be able to pass a timeout argument to the docker.APIClient constructor. We'd need a new module argument for the kolla_docker module in kolla-ansible.
[1] https:/