adding a 2nd website unit fails to call reverseproxy-relation-changed

[Bret Barker]

I'm trying to troubleshoot an issue where I've got a working relation-changed hook from my website providing charm to haproxy. All is good with the 1st deploy of both services, but when I add a 2nd website unit and it calls relation-set to the haproxy, the haproxy charm's reverseproxy-relation-changed never gets called.

juju deploy haproxy
juju deploy mywebsite
juju add-relation mywebsite:website haproxy:reverseproxy

everything works fine, mywebsite's website-relation-changed get's called, which does a (in ansible playbook):
  relation-set -r {{ item.__relid__ }} hostname={{ unit_private_address }} port={{ http_port }} service_name="my-site"

that correctly triggers haproxy's reverseproxy-relation-changed and the haproxy.cfg gets updated.

Then:

juju add-unit mywebsite

mywebsite's website-relation-changed get's called, which does the relation-set again, but this time haproxy never gets a hook called. I tried doing debug-hooks on it and nothing.

I'm working on hunting this down and will provide more details as I can narrow it down.

[Bret Barker (noise)]

This must have been some bad juju state, because it was consistently happening but then after a system restart it's working as it should.

[Bret Barker (noise)]

I take it back, I've now determined that was being caused by setting global_debug: 'true' on the haproxy charm.

[Bret Barker (noise)]

Repro steps:

1. use juju-deployer w/attached config
   juju-deployer -e local haproxy-test.cfg
2. check haproxy servers:
   juju ssh haproxy/0 grep server /etc/haproxy/haproxy.cfg
3. notice 2 meteor server lines
4. change global_debug to true,
5. nuke the env and redeploy
6. check again:
   juju ssh haproxy/0 grep server /etc/haproxy/haproxy.cfg
7. notice only 1 server line and 2nd reverseproxy-relation-changed never called on haproxy

Note this happens with any website-providing charm, just using meteor as an example.

[Bret Barker (noise)]

Even simpler example with pubphoto charm instead of meteor.

[Bret Barker (noise)]

I can't see any reason in the haproxy charm's hook code that would cause this, all it does w/that flag is write "debug" into the haproxy.cfg.

LazyPower - any ideas?

[Charles Butler (lazypower)]

@noise - I haven't had a chance to dig into this but will put it on my docket for this week.

[Bret Barker (noise)]

@lazyPower: cool, even if you can repro it so I know I'm not just crazy.

[Charles Butler (lazypower)]

@noise - yeah this is confirmed behavior. :(

I haven't had a chance circle back and work on this item - but i've updated the bug to confirmed. This is indeed side-effect behavior of something going awry in the charm