Comment 8 for bug 1629797

Revision history for this message
Dan Watkins (oddbloke) wrote : Re: cloud-init seems to take most of yakkety slow boot time

The above log and tests were on EC2.

On GCE, an instrumented xenial gives:

Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceGCE.DataSourceGCE'>
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Checking DNS redirection...
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Trying does-not-exist.example.com....
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Not redirected
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Trying example.invalid....
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Not redirected
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Trying KgWv29ZuRsiJoFAGfGdwRj6To1L4b44q...
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Not redirected
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: DNS redirection checking complete.
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Doing actual check...
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] util.py[DEBUG]: Actual check successful.
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] url_helper.py[DEBUG]: [0/1] open 'http://metadata.google.internal/computeMetadata/v1/instance/id' with {'method': 'GET', 'url': 'http://metadata.google.internal
Oct 03 09:40:43 xenial-161003-0956 cloud-init[1318]: [CLOUDINIT] url_helper.py[DEBUG]: Read from http://metadata.google.internal/computeMetadata/v1/instance/id (200, 19b) after 1 attempts

And an instrumented yakkety gives:

Oct 03 09:41:19 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] __init__.py[DEBUG]: Seeing if we can get any data from <class 'cloudinit.sources.DataSourceGCE.DataSourceGCE'>
Oct 03 09:41:19 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Checking DNS redirection...
Oct 03 09:41:19 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Trying does-not-exist.example.com....
Oct 03 09:41:44 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Not redirected
Oct 03 09:41:44 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Trying example.invalid....
Oct 03 09:42:10 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Not redirected
Oct 03 09:42:10 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Trying DLMcOj0Uix9neTVLk3ksle1KNUDrSz5p...
Oct 03 09:42:35 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Not redirected
Oct 03 09:42:35 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: DNS redirection checking complete.
Oct 03 09:42:35 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Doing actual check...
Oct 03 09:42:35 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] util.py[DEBUG]: Actual check successful.
Oct 03 09:42:35 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] url_helper.py[DEBUG]: [0/1] open 'http://metadata.google.internal/computeMetadata/v1/instance/id' with {'url': 'http://metadata.google.internal/computeMetadata
Oct 03 09:42:35 yakkety-161003-0945 cloud-init[1274]: [CLOUDINIT] url_helper.py[DEBUG]: Read from http://metadata.google.internal/computeMetadata/v1/instance/id (200, 19b) after 1 attempts

So it definitely looks like DNS resolution failure taking 25 seconds is A Thing (TM). It's also taking 25 seconds on both EC2 and GCE, which suggests the issue is somewhere in Ubuntu. It also is extremely unlikely to be that networking isn't up at all, because the successful check takes no time at all across several runs.