If bootstrap node is stopped in EC2, its address changes and all agents lose track of it

[Clint Byrum]

With bug #863526 fixed, we can shut down or reboot almost all machines and they will come back and re-join juju. But if the bootstrap node is down long enough where its address changes on EC2, the other machines will not be able to find it because their agents have the zookeeper address hard coded.

[Clint Byrum (clint-fewbar)]

Adding juju-core. I'd like to know if it is affected by this. Also marking the bug as Triaged, as I hit it again recently.

[Dave Cheney (dave-cheney)]

Won't fix. Not an issue

lucky(~/src/launchpad.net/juju-core) % juju status
machines:
  "0":
    agent-version: 1.9.4
    dns-name: ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com
    instance-id: i-314c3c0b
services: {}
lucky(~/src/launchpad.net/juju-core) % juju ssh 0 -- sudo shutdown -r now
Warning: Permanently added 'ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com,54.252.42.105' (ECDSA) to the list of known hosts.
Connection to ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com closed.
lucky(~/src/launchpad.net/juju-core) % juju status
machines:
  "0":
    agent-version: 1.9.4
    dns-name: ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com
    instance-id: i-314c3c0b
services: {}

[Kapil Thangavelu (hazmat)]

interesting, i hadn't realized that. http://aws.amazon.com/maintenance-help/
addresses will change with ec2-stop-instances / ec2-start-instances

On Wed, Dec 5, 2012 at 12:31 AM, Dave Cheney <email address hidden>wrote:

> Won't fix. Not an issue
>
> lucky(~/src/launchpad.net/juju-core) % juju status
> machines:
> "0":
> agent-version: 1.9.4
> dns-name: ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com
> instance-id: i-314c3c0b
> services: {}
> lucky(~/src/launchpad.net/juju-core) % juju ssh 0 -- sudo shutdown -r now
> Warning: Permanently added '
> ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com,54.252.42.105'
> (ECDSA) to the list of known hosts.
> Connection to ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.comclosed.
> lucky(~/src/launchpad.net/juju-core) % juju status
> machines:
> "0":
> agent-version: 1.9.4
> dns-name: ec2-54-252-42-105.ap-southeast-2.compute.amazonaws.com
> instance-id: i-314c3c0b
> services: {}
>
> ** Changed in: juju-core
> Status: New => Won't Fix
>
> ** Changed in: juju-core
> Milestone: None => 1.9.4
>
> --
> You received this bug notification because you are subscribed to juju.
> https://bugs.launchpad.net/bugs/963304
>
> Title:
> If bootstrap node is stopped or rebooted in EC2, its address changes
> and all agents lose track of it
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju/+bug/963304/+subscriptions
>

[Clint Byrum (clint-fewbar)]

This actually is still issue, not with reboot, but with stop/start it is an issue.

Seems to me the solution is to have state servers push their addresses into object storage at a well known location which is read before establishing a connection to the state server.

[Данило Шеган (danilo)]

This should be handled as part of the high availability work for juju-core: https://blueprints.launchpad.net/juju-core/+spec/s-cloud-juju-ha