Excellent breakdown of the bug scenario Scott, do you have any timeline on
the fixes for A, B or C?

I can update the customer based on your findings. Sorry for the lack of
responses all, I am still on-site working on other things.

Cheers,

- Calvin

On Mon, Feb 5, 2018 at 7:23 PM, Scott Moser <email address hidden>
wrote:

> > - This causes a new VM to be provisioned on VMware. Juju
> >   creates/uploads/deploys a vmdk file which is used to deploy the new
> >   VM.  This VMDK appears to be cached and is not generated everytime a
> >   host is provisioned for speed, so it must contain a static hostname.
>
> The OVF disk contains a hostname... this is very much the platform
> telling cloud-init what the hostname should be.
>
> Lets call this 'platform-provided hostname'
>
> > The third step is where this breaks, because for some reason every VM on
> > boot has the hostname ubuntu, even though juju is passing the correct
> > hostname to cloud-init during boot, so the DNS server has a ton of IP
> > address entries for 'ubuntu' all relating to different hosts.
>
> I assume that juju is sending this in cloud-config inside user-data,
> the key being essentially:
>   #cloud-config
>   hostname: foo
>
> Lets call this 'user-provided hostname' as it comes in user-data.
>
> every VM has the hostname 'ubuntu' in /etc/hostname. Cloud-init does
> not change this until after the that initial dhcp is done.  As you've
> found a reboot (or "bounce" of the network device) would have dhcp
> re-read the hostname and update the dns records.
>
> The user-provided should (and does) override the platform-provided
> hostname.
>
> We have basically 3 possible sources of hostname information:
> a.) platform-provided (meta-data)
> b.) direct user-provided user-data
> c.) indirect user-provided hostname.
> I assume juju doesn't do this, but it could provide user-data that
> consisted of:
>   #include http://some.resource/some/path
>   #include http://some.other.resource/some/path
>
> where one of the the urls there contains:
>    #cloud-config
>    hostname: foo
>
> cloud-init doesn't currently process *any* user-data until after
> networking is configured.
>
> We have a fairly straight forward plan to fix 'a' with some re-factoring.
> Fixing 'b' requires more re-factoring, essentially "peeking" at user-data
> before all network resources are up.
> Fixing 'c' is more work, and would probably involve cloud-init somehow
> bouncing interfaces, or some other mechanism for convincing dhcp to
> update its lease information.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1746455
>
> Title:
>   cloud-init vSphere cloud provider DHCP unique hostname issue
>
> Status in cloud-init:
>   Incomplete
>
> Bug description:
>   We've had an issue recently when provisioning workloads on vSphere.
>   Here is what happens:
>
>   1) Set vSphere as the cloud provider in juju (bootstrap)
>
>   2) Deploy workload on vSphere, in our case, Canonical Kubernetes (CDK)
>
>   3) Machines are deployed using VMDK, but when they boot the initial
>   hostname is Ubuntu (for all machines) until juju changes it during
>   deployment.
>
>   4) DHCP gives each machine a lease, but the DNS entries are all messed
>   up because of identical hostnames.
>
>   5) If the machines are rebooted they will receive the correct DNS
>   entries. Note that juju will set the correct hostname eventually, but
>   not when the initial DHCP lease is given to the host.
>
>
>   Basically cloud-init is not giving newly provisioned machines on vSphere
> their correct hostname before the DHCP lease is given, meaning that every
> instance during DHCP has the hostname 'ubuntu' which messes up the entries
> on the DNS server.
>
>   Related Bugs:
>
>    * https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1600766
>    * https://github.com/juju-solutions/bundle-canonical-
> kubernetes/issues/480
>    * https://github.com/juju-solutions/bundle-canonical-
> kubernetes/issues/171
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-init/+bug/1746455/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=cloud-init; status=Incomplete; importance=High;
> <email address hidden>;
> Launchpad-Bug-Tags: cdo-qa-blocker cloud-init cpe-onsite dhcp juju k8s
> vsphere
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: calvinh davidpbritton jameinel smoser thumper
> Launchpad-Bug-Reporter: Calvin Hartwell (calvinh)
> Launchpad-Bug-Modifier: Scott Moser (smoser)
> Launchpad-Message-Rationale: Subscriber
> Launchpad-Message-For: calvinh
>



-- 
E:  <email address hidden>
W: http://calvinhartwell.com
T:  (+44) 07534801542