When installing a node, euca_find_cluster fails to locate the cluster controller if instances are running

[Nick Barcet]

Paul Gyermonprez from Intel, reported the following bug for which I do not have the necessary hw to try to reproduce:

Adding nodes in several waves after launching VMs is not possible :

1. The “node mode” is discovered and the step is hanging
2. This step is the automatic preseed
3. But the answer from the server (via avahi ?) is sending a link to a wrong IP.
4. This wrong IP is the last one added to the eth interface.and the last one is the vlan used for VM communication

Workaround : add ALL nodes before you launch the first VM
Or close all VMs and make sure the only IPv4 for your eth is the one you want

==================
Karmic Release notes:

UEC Node Controller installation failure in an existing UEC

Extending an existing Ubuntu Enterprise Cloud may fail during node controller installation started from the ISO with the "Install Ubuntu Enterprise Cloud" option. The Node installation fails with an error message stating that the preseed file cannot be downloaded from the Cluster Controller as the wrong IP address is used to connect to the Cluster Controller.

The workaround is to perform a standard package installation rather than an ISO installation: Install a default Karmic server and then install the eucalyptus-nc package. Additionally the system's primary ethernet interface needs to be configured as a bridge and the Cloud Controller's eucalyptus user's public ssh key needs to be copied into the Node Controller's eucalyptus user's authorized_keys file. More detailed steps can be found in the UEC Package Install tutorial, Step 3 at https://help.ubuntu.com/community/UEC/PackageInstall.

==================
Avahi SRU

Impact:
Public IPs are added to the Cluster Controller (CC) when instances are booted. These IPs are then automatically published by the avahi-daemon running on the CC. The instance public IPs should not be resolvable via avahi.

How the bug has been addressed:
The default avahi-daemon upstart job has been modified to use /etc/eucalyptus/avahi-daemon.conf (which will be shipped by the eucalyptus package) as the configuration file instead of the default configuration file.

To reproduce the bug:
1. Install avahi-daemon and make sure it's started with the default configuration file (ie no configuration file specified on the command line - the process should use the hostname) - both an a new install and on upgrades:

  mathiaz@t-avahi:~$ ps -ef | grep avahi
  avahi 1373 1 0 11:30 ? 00:00:00 avahi-daemon: running [t-avahi.local]
  avahi 1375 1373 0 11:30 ? 00:00:00 avahi-daemon: chroot helper
  mathiaz 1422 977 0 11:31 pts/0 00:00:00 grep avahi

2. Copy /etc/avahi/avahi-daemon.conf to /etc/eucalyptus/avahi-daemon.conf and edit /etc/eucalyptus/avahi-daemon.conf to change the published hostname to eucalyptus. Restart avahi-daemon. Make sure the process has been restarted and running with the new eucalyptus avahi-daemon configuration file:

  mathiaz@t-avahi:~$ ps -ef | grep avahi
  avahi 2070 1 1 11:35 ? 00:00:00 avahi-daemon: running [eucalyptus.local]
  avahi 2071 2070 0 11:35 ? 00:00:00 avahi-daemon: chroot helper
  mathiaz 2073 977 0 11:35 pts/0 00:00:00 grep avahi

Regression potential:
eucalyptus is not installed by default - avahi-daemon should just restart normally. On existing eucalyptus Cluster Controllers, resolution via avahi for the CC will not work until the eucalyptus-cc-publication-ip job has started.

==================
eucalyptus sru

Impact:
Adding new Node Controllers by installing them via the -server ISO will fail if instances with public IPs are running. This is due to the fact that public IPs are added on the Cluster Controller whose avahi-daemon will automatically publish. Since the installer relies on Avahi resolution to find the IP address of the Cluster Controller, the install will fail while trying to download the preseed file from the last public IP added to the CC.

How the bug has been addressed:
A specific avahi daemon configuration has been added that will not publish addresses by default - instead a specific upstart job has been added that will publish the CC IPs.

To reproduce the bug:
1. Install a Cluster Controller and one Node Controller. Start an instance with a public IP and note its public IP.
2. Boot from the -server iso and select to install an Ubuntu Entreprise Cloud.
3. Before the update, the installation process will fail with an error message stating that the preseed file cannot be downloaded from the public IP address of the instance.
4. After the update on the CC, the installation process is successful (ie the preseed file is correctly downloaded from the CC IP).

Regression potential:
The avahi-daemon running on the CC will stop publishing any IP addresses configured on any interfaces from the CC. Connection from a local network relying on avahi dns resolution will be broken. Note that it only affects system which have the eucalyptus-cc package installed.

==================

[Thierry Carrez (ttx)]

If true this is nasty because adding more nodes on an existing cloud is a very common use case.
Could anyone with the necessary hardware (multinode setup) reproduce that ?

[Mathias Gug (mathiaz)]

I've confirmed the bug.

euca_find_cluster returns the last public IP address allocated to an instance instead of the IP address published by avahi-publish on the CC.

[Mathias Gug (mathiaz)]

On the CC:

ubuntu@uec-cc:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1c:c4:6b:38:4a brd ff:ff:ff:ff:ff:ff
    inet 10.153.108.210/24 brd 10.153.108.255 scope global eth0
    inet 192.168.222.10/32 scope global eth0:pub
    inet 192.168.222.13/32 scope global eth0:pub
    inet 192.168.222.11/32 scope global eth0:pub
    inet 192.168.222.12/32 scope global eth0:pub
    inet6 fe80::21c:c4ff:fe6b:384a/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1c:c4:6b:38:48 brd ff:ff:ff:ff:ff:ff
    inet 169.254.169.254/32 scope link eth1:metadata
    inet 192.168.142.2/24 brd 192.168.142.255 scope global eth1
    inet 172.19.1.1/27 brd 172.19.1.31 scope global eth1:priv
    inet6 fe80::21c:c4ff:fe6b:3848/64 scope link
       valid_lft forever preferred_lft forever

ubuntu@uec-cc:~$ ps -ef | grep publish
root 1856 1 0 15:20 ? 00:00:00 avahi-publish -s 10.153.108.210 _eucalyptus._tcp 8774 txtvers=1 protovers=1.5.0 type=cluster
ubuntu 10815 3054 0 16:27 pts/0 00:00:00 grep --color=auto publish

On the NC in the installer, euca_find_cluster returns:

192.168.222.12:8774

[Mathias Gug (mathiaz)]

According to euca_find_cluster source code the service name published by avahi-publish on the CC is only used if the IP address returned by avahi is 169.254.169.254:

            if (avahi_address_snprint(human_address, AVAHI_ADDRESS_STR_MAX, address)) {
                if (address->proto == AVAHI_PROTO_INET6)
                    key = avahi_strdup_printf("[%s]:%u", human_address, port);
                else if (strncmp(human_address, "169.254.169.254", 15) == 0)
                    key = avahi_strdup_printf("%s:%u", name, port);
                else
                    key = avahi_strdup_printf("%s:%u", human_address, port);

[Mathias Gug (mathiaz)]

A standard ubuntu-server installation followed by a package installation (sudo apt-get install eucalyptus-nc - as outlined in https://help.ubuntu.com/community/UEC/PackageInstall) works correctly.

This issue is only encounter for UEC ISO installation.

[Thierry Carrez (ttx)]

We need to understand what's going on exactly : if the bug is in the installer and there is no way to workaround it by a future CC upgrade, we might need to fix the installer itself.

[Mathias Gug (mathiaz)]

After more investigation it turns out that the problem lies with the avahi resolver. Whenever a new public IP address (for an instance) is added to the CC, avahi-daemon will publish it automatically.

Subsequent installs will use one of the publish IP by avahi daemon (which turns out to be the last IP added to the public network interface). euca_find_cluser uses avahi dns resolution to get the CC IP address to get the seed from and will only use the published IP by the CC avahi publication job if the resolution leads to 169.254.169.254.

The proposed workaround is to always use the IP publish by the CC publication job.

[Mathias Gug (mathiaz)]

I can confirm that the proposed branch fixes the issue:

1. Make a local build and extract euca_find_cluster from the eucalyptus udeb.
2. Copy euca_find_cluster to http://people.canonical.com/~mathiaz/euca_find_cluster
3. Start a few instances on the UEC install and make sure that the public IPs are configured:

ubuntu@uec-cc:~$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1c:c4:6b:38:4a brd ff:ff:ff:ff:ff:ff
    inet 10.153.108.210/24 brd 10.153.108.255 scope global eth0
    inet 192.168.222.11/32 scope global eth0:pub
    inet 192.168.222.15/32 scope global eth0:pub
    inet 192.168.222.17/32 scope global eth0:pub
    inet6 fe80::21c:c4ff:fe6b:384a/64 scope link
       valid_lft forever preferred_lft forever

4. Boot a karmic server install and select Install Ubuntu Enterprise Cloud.
5. Once the network is up and the installer is prompting for the hostname open a new shell.
6. Dowload the new euca_find_cluster.
7. Verify that the normal euca_find_cluster returns 192.168.222.17 as the IP address of the CC.
8. Verify that the new euca_find_cluster returns 10.153.108.210 as the IP address of the CC.
9. Copy the new euca_find_cluster to /usr/bin/
10. Proceed with the installation which is successful.

[Mathias Gug (mathiaz)]

Another workaround is to explicitly publish the hostname/IP address mapping on the CC:

  avahi-publish -a $(hostname).local CC_IP

That way the avahi dns resolution will always resolve the hostname to the correct IP address.

[Mathias Gug (mathiaz)]

Here is an upstart job that does the hostname/IP publication:

description "Eucalyptus cluster controller IP publication"

start on started eucalyptus-cc
stop on stopping eucalyptus-cc

script
 . /etc/eucalyptus/eucalyptus.conf
 . /etc/eucalyptus/eucalyptus-ipaddr.conf
 exec avahi-publish -a $(hostname).local $CC_IP_ADDR
end script

[Thierry Carrez (ttx)]

I tested the proposed update-based solution and it does *not* seem to work:

* Added the avahi-publish upstart job
* Restart cluster
* Cluster has:
$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:13:20:f8:4d:11 brd ff:ff:ff:ff:ff:ff
    inet 169.254.169.254/32 scope link eth0:metadata
    inet 192.168.0.127/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::213:20ff:fef8:4d11/64 scope link
       valid_lft forever preferred_lft forever
* Launch UEC node install : euca_find_cluster says 192.168.0.127:8774
* On Cluster, add an IP address: sudo ip addr add 192.168.0.230 dev eth0 label eth0:pub
* Cluster now has:
$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:13:20:f8:4d:11 brd ff:ff:ff:ff:ff:ff
    inet 169.254.169.254/32 scope link eth0:metadata
    inet 192.168.0.127/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.230/32 scope global eth0:pub
    inet6 fe80::213:20ff:fef8:4d11/64 scope link
       valid_lft forever preferred_lft forever
* Launch UEC node install: euca_find_cluster now says 192.168.0.230:8774

You still have to explicitly disable IP address publication for it to work:
Add "publish-addresses=no" to [publish] section in avahi-daemon.conf
Then new addresses won't be published and resolution will always work.

[Thierry Carrez (ttx)]

Given that avahi-daemon ships with publish-addresses=yes, that may be difficult to fix in a SRU.

[Martin Pitt (pitti)]

How about this, for a h4ck1sh SRU:

 - Change avahi's init script to prefer /etc/eucalyptus/avahi-daemon.conf if it is present
 - Change euca package to ship this.

This would avoid euca having to change another packages' conffile (which is a grave policy violation for good reason), and should DTRT, does not change avahi's default behaviour, and the changes are small enough to be SRUable.

Would that work?

[Thierry Carrez (ttx)]

Yes it would.

Other SRU-based options have been considered in #ubuntu-release by mdz, mathiaz and kirkland, but were not mentioned here. That made me doubt that the solution presented in comment 10 was appropriate. In light of that discussion and Martin's solution, I think we can safely discuss the best fix postrelease.

[Mathias Gug (mathiaz)]

Shipping with publish-addresses=no won't fix the problem: euca_find_cluster will not be able resolve the IP address of the CC at all.

To confirm thierry's comment #11, I've tested the upstart job and it failed. avahi-publish refuses to start since the CC IP is already resolve to another name via avahi-daemon.

[Mathias Gug (mathiaz)]

Release notes:

Extending an existing Ubuntu Enterprise Cloud may fail during node controller installation started from the ISO with the "Install Ubuntu Enterprise Cloud" option. The Node installation fails with an error message stating that the preseed file cannot be downloaded from the Cluster Controller as the wrong IP address is used to connect to the Cluster Controller.

The workaround is to conduct a standard package installation rather than an ISO installation: Install a default Karmic server and then install the eucalyptus-nc package. Additionally the system's primary ethernet interface needs to be configured as a bridge and the Cloud Controller's eucalyptus user's public ssh key needs to be copied into the Node Controller's eucalyptus user's authorized_keys file. More detailed steps can be found in the UEC Package Install tutorial, Step 3 at https://help.ubuntu.com/community/UEC/PackageInstall.

[Thierry Carrez (ttx)]

Mathias, re: comment 15:
To fix this you need to manually publish the address (your upstart script) *and* run avahi with publish-addresses=no. That way the right mapping address is published and the new ones don't get auto-published.

[Steve Langasek (vorlon)]

Documented at <https://wiki.ubuntu.com/KarmicKoala/ReleaseNotes#UEC%20Node%20Controller%20installation%20failure%20in%20an%20existing%20UEC>:

Extending an existing Ubuntu Enterprise Cloud may fail during node controller installation started using the "Install Ubuntu Enterprise Cloud" option on the server CD. The node installation reports that the preseed file cannot be downloaded from the Cluster Controller, because the wrong IP address is used to connect to the Cluster Controller.

As a workaround for this issue, users can install a standard Ubuntu 9.10 server and then install the {{{eucalyptus-nc}}} package after reboot. Additionally, the system's primary ethernet interface will need to be configured as a bridge and the public ssh key of the Cloud Controller's eucalyptus user will need to be manually copied into the {{{authorized_keys}}} file of the Node Controller's eucalyptus user. More detailed instructions can be found in Step 3 of the UEC Package Install tutorial at [[https://help.ubuntu.com/community/UEC/PackageInstall]]. (Bug:458904)

[Mathias Gug (mathiaz)]

The workaround discussed by Thierry and Martin in comment 11-14 is working as expected:

1. Update avahi-daemon upstart job to use /etc/eucalytpus/avahi-daemon.conf if is there:

=== modified file 'debian/avahi-daemon.upstart'
--- debian/avahi-daemon.upstart 2009-09-15 03:33:13 +0000
+++ debian/avahi-daemon.upstart 2009-10-30 21:07:05 +0000
@@ -12,4 +12,8 @@
 expect daemon
 respawn

-exec avahi-daemon -D
+script
+ opts="-D"
+ [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"
+ exec avahi-daemon ${opts}
+end script

2. Ship /etc/eucalyptus/avahi-daemon.conf in the eucalyptus-cc package that disables address publishing:

publish-addresses=no

3. Add an eucalyptus-cc-publication-ip upstart job to publish the CC IP/hostname mapping:

exec avahi-publish -a $(hostname).local $CC_IP_ADDR

With these changes, running new instances with public IPs doesn't change the IP address returned by euca_find_cluster for an NC iso installation.

[Martin Pitt (pitti)]

Mathias Gug [2009-10-30 21:52 -0000]:
> -exec avahi-daemon -D
> +script
> + opts="-D"
> + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"

Upstart scripts are "set -e" by default, so you are missing an "||
true" here. But even more, if /etc/eucalyptus/avahi-daemon.conf does
not exist, you need to drop -D entirely (to use the default file). So
I think you want

> -exec avahi-daemon -D
> + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && conf="-f /etc/eucalyptus/avahi-daemon.conf" || :
> + exec avahi-daemon -D $conf
> +end script

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Steve Langasek (vorlon)]

On Sat, Oct 31, 2009 at 12:40:46PM -0000, Martin Pitt wrote:
> Mathias Gug [2009-10-30 21:52 -0000]:
> > -exec avahi-daemon -D
> > +script
> > + opts="-D"
> > + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"

> Upstart scripts are "set -e" by default, so you are missing an "||
> true" here.

$ set -e
$ [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"
$ echo $?
1
$

> But even more, if /etc/eucalyptus/avahi-daemon.conf does
> not exist, you need to drop -D entirely (to use the default file). So
> I think you want

> > -exec avahi-daemon -D
> > + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && conf="-f /etc/eucalyptus/avahi-daemon.conf" || :
> > + exec avahi-daemon -D $conf
> > +end script

That doesn't appear to drop the -D at all? I guess maybe you mean:

script
 conf=-D
 [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && conf="-f /etc/eucalyptus/avahi-daemon.conf"
 exec avahi-daemon $conf
end script

?

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Martin Pitt (pitti)]

Steve Langasek [2009-11-02 11:06 -0000]:
> > > + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"
>
> > Upstart scripts are "set -e" by default, so you are missing an "||
> > true" here.
>
> $ set -e
> $ [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"
> $ echo $?
> 1

(thanks for the confirmation)

> > > -exec avahi-daemon -D
> > > + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && conf="-f /etc/eucalyptus/avahi-daemon.conf" || :
> > > + exec avahi-daemon -D $conf
> > > +end script
>
> That doesn't appear to drop the -D at all? I guess maybe you mean:

No, it shouldn't. -D means "daemonize", and this should always be
done. What we want to do is to add -f /my/different/conffile.

> script
> conf=-D
> [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && conf="-f /etc/eucalyptus/avahi-daemon.conf"
> exec avahi-daemon $conf
> end script

That would start avahi-daemon in non-daemon mode for euca; this might
work with upstart (haven't checked), but I advise against making such
changes in an SRU.

Martin

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

[Steve Langasek (vorlon)]

On Mon, Nov 02, 2009 at 12:13:28PM -0000, Martin Pitt wrote:
> Steve Langasek [2009-11-02 11:06 -0000]:
> > > > + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"
> >
> > > Upstart scripts are "set -e" by default, so you are missing an "||
> > > true" here.

> > $ set -e
> > $ [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && opts="${opts} -f /etc/eucalyptus/avahi-daemon.conf"
> > $ echo $?
> > 1

> (thanks for the confirmation)

That's not a confirmation, I'm demonstrating that checking the return value
with '&&' is sufficient to prevent the shell from exiting under set -e. So
no '|| true' is needed.

> > > > -exec avahi-daemon -D
> > > > + [ -e "/etc/eucalyptus/avahi-daemon.conf" ] && conf="-f /etc/eucalyptus/avahi-daemon.conf" || :
> > > > + exec avahi-daemon -D $conf
> > > > +end script

> > That doesn't appear to drop the -D at all? I guess maybe you mean:

> No, it shouldn't. -D means "daemonize", and this should always be
> done. What we want to do is to add -f /my/different/conffile.

Ok, then I guess I misunderstood what you meant when you said

> if /etc/eucalyptus/avahi-daemon.conf does not exist, you need to drop
> -D entirely (to use the default file).

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Martin Pitt (pitti)]

> checking the return value with '&&' is sufficient to prevent the shell from exiting under set -e

Ah, thanks for pointing out.

> if /etc/eucalyptus/avahi-daemon.conf does not exist, you need to drop
> -D entirely (to use the default file).

Sorry, that was BS, ignore me.

So the original patch in comment 19 is just fine.

[Martin Pitt (pitti)]

Accepted avahi into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Martin Pitt (pitti)]

Mathias - is the avahi change a hack for karmic only, or should be proliferated to lucid? I. e. is this "fix committed" or "wontfix" for lucid?

[Mathias Gug (mathiaz)]

Good question for lucid. I don't know yet. I've got another proposal for lucid that involves not shipping a specific configuration for avahi daemon (the proposal is to use a txt record to publish the CC IP - the same way as the type - cluster or node - is publish currently. That would require some changes in the installer).

So I'll mark the bug as Wontfix in avahi for now as the proposal doesn't involve any changes to the avahi package.

[Martin Pitt (pitti)]

Accepted eucalyptus into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Thierry Carrez (ttx)]

This is *NOT* fixed in the version in proposed.
There is no avahi-daemon.conf shipped in /etc/eucalyptus (probably missing some eucalyptus-cc.install change), so regular address autopublication is still occurring.

[Dustin Kirkland  (kirkland)]

Okay, I fixed this in a .2 upload.

Debdiff against the .1 version in karmic-proposed attached for SRU team's review.

I verified that avahi-daemon.conf actually lands in the eucalyptus-cc deb, and gets installed to:
-rw-r--r-- root/root 620 2009-11-10 14:00 ./etc/eucalyptus/avahi-daemon.conf

[Martin Pitt (pitti)]

Accepted .2, please test once it's built.

[Thierry Carrez (ttx)]

This is now working, *but* upgrading to the latest package is not enough to fix it on a running system. You need to restart avahi-daemon (to make it pick up the new avahi-daemon.conf) before you restart eucalyptus (and the publication takes place). Rebooting would also work, as it would restart both.

upgrade to 7.2 (which restarts eucalyptus): test = FAIL
  then restart avahi-daemon : test = FAIL
  then restart eucalyptus : test = PASS

upgrade to 7.2, reboot: test = PASS

If that's not acceptable, I'd suggest postinst should pick up that upgrade path and restart avahi before the eucalyptus publication jobs are restarted.

[Mathias Gug (mathiaz)]

Right. However the eucalyptus update should be applied at the same time as the avahi-daemon update. In such a scenario the daemon will be restarted by the avahi-daemon ugprade and the new configuration file should be picked up. I'll test this use case and report back.

[Mathias Gug (mathiaz)]

Ok - that doesn't work either: conffiles are installed during the configuration step (before postinst is called) rather than then the unpack phase. As such eucalyptus specific avahi configuration file is not available when avahi-daemon is restarted by avahi-daemon postinstall script.

avahi-daemon needs to be explicitly restarted by eucalyptus-postinst script.

[Martin Pitt (pitti)]

Accepted eucalyptus into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Mathias Gug (mathiaz)]

Tested 7.3 upgrade and everything works as expected. New instances with public IPs will not change the address resolution.

[Launchpad Janitor (janitor)]

This bug was fixed in the package avahi - 0.6.25-1ubuntu5.1

---------------
avahi (0.6.25-1ubuntu5.1) karmic-proposed; urgency=low

  * debian/avahi-daemon.upstart: use eucalyptus specific avahi-daemon
    configuration file if it's available. (LP: #458904)

 -- Mathias Gug <email address hidden> Fri, 30 Oct 2009 17:43:22 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package eucalyptus - 1.6~bzr931-0ubuntu7.3

---------------
eucalyptus (1.6~bzr931-0ubuntu7.3) karmic-proposed; urgency=low

  * debian/eucalyptus-cc.postinst: restart avahi daemon so that it uses
    eucalyptus specific configuration file (LP: #458904).
  * debian/eucalyptus-cc.eucalyptus-cc-publication{,-ip}.upstart: Respawn
    avahi publication jobs if they die (LP: #480885).

eucalyptus (1.6~bzr931-0ubuntu7.2) karmic-proposed; urgency=low

  [ Dustin Kirkland ]
  * cluster/handlers.c: euca_rootwrap rework did not whitelist powerwake;
    however, powerwake does *not* need root privs, drop euca_rootwrap wrapper
    (LP: #458163)
  * debian/rules, debian/euclayptus-cc.install: install the avahi-daemon.conf
    in /etc/eucalyptus, (LP: #458904).

  [ Thierry Carrez ]
  * clc/modules/www/src/main/java/edu/ucsb/eucalyptus/admin/public/EucalyptusWebInterface.html:
    Fix HTML title in the web UI for more consistency in naming (LP: #455293)
  * debian/eucalyptus-common.eucalyptus.upstart: Add -l to eucalyptus-cloud
    options so that cloud-output.log is affected by LOGLEVEL (LP: #458001)

  [ Mathias Gug ]
  * cluster/handlers.c: Fix the networkIndex returned by describeInstances.
    (LP: #454405 - upstream revno 933).
  * debian/eucalyptus-cc.eucalyptus-cc-publication{,-ip}.upstart: add an
    upstart job to explicitly publish the IP/CC hostname mapping via avahi
    instead of publishing the CC IP address via the service name (LP: #458904).
  * debian/avahi-daemon.conf: ship a specific avahi-daemon configuration file
    that doesn't publish IP addresses by default. (LP: #458904).
  * debian/eucalyptus-cloud.postinst: Fix postfix configuration to accept
    confirmation emails sent by eucalyptus (LP: #459101)
  * debian/eucalyptus-cc.upstart: Don't clean the CC network state when the CC is
    stopped by default (LP: #460089).
 -- Mathias Gug <email address hidden> Wed, 11 Nov 2009 15:15:48 -0500

[Martin Pitt (pitti)]

I copied the karmic-proposed package to lucid. Ffor karmic-updates it is still missing two verifications.

[Colin Watson (cjwatson)]

Reopening for Lucid as we want to fix this a different way there (with an explicit TXT record).

[Launchpad Janitor (janitor)]

This bug was fixed in the package eucalyptus - 1.6~bzr931-0ubuntu7.3

---------------
eucalyptus (1.6~bzr931-0ubuntu7.3) karmic-proposed; urgency=low

  * debian/eucalyptus-cc.postinst: restart avahi daemon so that it uses
    eucalyptus specific configuration file (LP: #458904).
  * debian/eucalyptus-cc.eucalyptus-cc-publication{,-ip}.upstart: Respawn
    avahi publication jobs if they die (LP: #480885).

eucalyptus (1.6~bzr931-0ubuntu7.2) karmic-proposed; urgency=low

  [ Dustin Kirkland ]
  * cluster/handlers.c: euca_rootwrap rework did not whitelist powerwake;
    however, powerwake does *not* need root privs, drop euca_rootwrap wrapper
    (LP: #458163)
  * debian/rules, debian/euclayptus-cc.install: install the avahi-daemon.conf
    in /etc/eucalyptus, (LP: #458904).

  [ Thierry Carrez ]
  * clc/modules/www/src/main/java/edu/ucsb/eucalyptus/admin/public/EucalyptusWebInterface.html:
    Fix HTML title in the web UI for more consistency in naming (LP: #455293)
  * debian/eucalyptus-common.eucalyptus.upstart: Add -l to eucalyptus-cloud
    options so that cloud-output.log is affected by LOGLEVEL (LP: #458001)

  [ Mathias Gug ]
  * cluster/handlers.c: Fix the networkIndex returned by describeInstances.
    (LP: #454405 - upstream revno 933).
  * debian/eucalyptus-cc.eucalyptus-cc-publication{,-ip}.upstart: add an
    upstart job to explicitly publish the IP/CC hostname mapping via avahi
    instead of publishing the CC IP address via the service name (LP: #458904).
  * debian/avahi-daemon.conf: ship a specific avahi-daemon configuration file
    that doesn't publish IP addresses by default. (LP: #458904).
  * debian/eucalyptus-cloud.postinst: Fix postfix configuration to accept
    confirmation emails sent by eucalyptus (LP: #459101)
  * debian/eucalyptus-cc.upstart: Don't clean the CC network state when the CC is
    stopped by default (LP: #460089).
 -- Mathias Gug <email address hidden> Wed, 11 Nov 2009 15:15:48 -0500

[Colin Watson (cjwatson)]

eucalyptus (1.6.1~bzr1083-0ubuntu1) lucid; urgency=low

  [ Dustin Kirkland ]
  * Merge upstream bzr revision 1082; the following bugs have been fixed
    upstream since the last merge:
    - LP: #378969 - private bug
    - LP: #404842 - init script fix
    - LP: #434283 - existing keys should be overwritten unconditionally
    - LP: #445990 - run instance will fail if no kernel or ramdisk specified
    - LP: #447457 - euca_conf --register-sc ... check the number of parameters
    - LP: #449874 - fix incorrect help text (--delete-nodes doesn't exist)
    - LP: #451795 - show registered images in elastic fox
    - LP: #454405 - return correct networkIndex values on describeInstances
    - LP: #456877 - init script fix
    - LP: #456878 - fix for libvirt xen driver
    - LP: #460085 - fix rampart memory leak
    - LP: #461156 - fix authentication problem w/ userdata
    - LP: #461394 - fix multiple concurrent snapshots on the same volume
    - LP: #461444 - fix memory leaks in NC getConsoleOutput and startup_thread
    - LP: #469984 - fix iptables rules issue
    - LP: #477776 - fix query string authentication
    - LP: #480783 - allow api connection over https
    - LP: #482249 - fix "Describe Regions"
    - LP: #484217 - create keypair should return an error if key exists
    - LP: #490623 - parse RFC 1123 formatted datetime
  * debian/control:
    - make all package lists one-per-line (makes changes henceforth more
      readable), sort lists
    - depend on rampart >= 1.3.0-0ubuntu6, which fixes some shared library
      installation issues
  * debian/patches/04-axis2c-1.6.0-rampart-1.3.0.patch: drop this patch,
    since Eucalyptus 1.6.1 natively supports axis2c 1.6.0 now
  * debian/eucalyptus-cloud.install,
    debian/eucalyptus-common.eucalyptus.upstart,
    debian/eucalyptus-java-common.install, debian/eucalyptus-sc.install,
    debian/eucalyptus-walrus.install: update static version number strings
    from "1.6-devel" to "1.6.1"; (we should really find a better way to do
    this)
  * debian/patches/03-DESTDIR.patch: ported forward for merge

 -- Dustin Kirkland <email address hidden> Tue, 01 Dec 2009 21:09:28 -0600

eucalyptus (1.6~bzr931-0ubuntu9) UNRELEASED; urgency=low

  [ Dustin Kirkland ]
  * cluster/handlers.c: euca_rootwrap rework did not whitelist powerwake;
    however, powerwake does *not* need root privs, drop euca_rootwrap wrapper
    (LP: #458163)
  *

  [ Thierry Carrez ]
  * clc/modules/www/src/main/java/edu/ucsb/eucalyptus/admin/public/EucalyptusWebInterface.html:
    Fix HTML title in the web UI for more consistency in naming (LP: #455293)
  * debian/eucalyptus-common.eucalyptus.upstart: Add -l to eucalyptus-cloud
    options so that cloud-output.log is affected by LOGLEVEL (LP: #458001)

  [ Colin Watson ]
  * Move cluster private interface selection forward to just after network
    configuration (LP: #455816).
  * Add the CC's IP address as an explicit TXT record in the Avahi
    advertisement, and make use of it in euca_find_cluster (LP: #458904).
  * Advertise the CLC and Walrus using Avahi.
  * Rename euca_find_cluster to euca_find_component and make...