HTTPS connection failing for Docker >= 1.10

[Michael Stieler]

We experience problems with outgoing HTTPS connections from Docker containers when running in OpenStack VMs.

We assume this could be a bug in OpenStack because:
- Ubuntu 14, 16 and CoreOS show the same problems
- While there are no problems with Docker 1.6.2 and 1.9.1, it fails with Docker 1.10 and 1.11
- The same containers work outside OpenStack
- We found similar problem descriptions in the web that occured on other OpenStack providers

The issue can easily be reproduced with:

1.) Installing a docker version >= 1.10
2.) docker run -it ubuntu apt-get update

Expected output: Ubuntu updates its package list
Actual output: Download does not start and runs into a timeout

The same problem seems to occur with wget and curl and our Java application.

Please note that plain HTTP works as expected, so does issuing the Https requests from the Docker host machine.

Disabling network virtualization with Docker flag --net="host" fixes the problems with wget, curl and apt-get, unfortunately not with the Java app we're trying to deploy in OpenStack.

For our current project this is actually a blocker since CoreOS comes bundled with a recent Docker version which is not trivial to downgrade.

I can't see any version information in the Horizon interface of our provider, however I think I heard they are using Mitaka release.

Links:
- Related issue at Docker: https://github.com/docker/docker/issues/20178
- ServerFault question by me: http://serverfault.com/questions/785768/https-request-fails-in-docker-1-10-with-virtualized-network
- StackOverflow question by someone else: http://stackoverflow.com/questions/35300497/docker-container-not-connecting-to-https-endpoints

[John Davidge (john-davidge)]

Hi Michael, thanks for the report. While I try to get some eyes on this have you tried reaching out for support on https://ask.openstack.org/en/questions/ ?

[Michael Stieler (1-michael-6)]

Hi, thanks for the quick response. I cross-posted this to the OpenStack Q&A right now and am now awaiting moderator approval.

[Michael Stieler (1-michael-6)]

Hi,

I was given a good advice from the Docker community. The problem is actually related to an incompatible MTU setting AND in my opinion not a problem of OpenStack at all.

As shown in https://github.com/docker/docker/issues/22028 Docker used to infer the MTU settings from the host network card and changed that behaviour to use 1500 by default starting from version 1.10.

OpenStack seems to run with MTU:1454, so it is now neccesary to configure Docker daemon to use that MTU also for the container's NICs. There is a blog post describing how to accomplish that:

https://rahulait.wordpress.com/2016/02/28/modifying-default-mtu-for-docker-containers/

We followed that, using 1454 instead of 1450 and HTTPs connections now work.

Sorry for filing a bug here, maybe it is worth mentioning in OpenStack docs that users should be aware of the MTU setting (if not already done).