Hook errors when config contains backends that haven't been related yet

Bug #1779747 reported by Casey Marshall
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
charm-haproxy
Triaged
High
Unassigned

Bug Description

haproxy hooks can fail if the services YAML config doesn't match the relations. The charm seems to assume 'mode tcp' and this can conflict with what relations specify in their relation data. For example:

2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : http frontend 'omnibus-public-2-81' (/etc/haproxy/haproxy.cfg:50) tries t
o use incompatible tcp backend 'directory_public' (/etc/haproxy/haproxy.cfg:149) as its default backend (see 'mode').
2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : http frontend 'omnibus-public-2-81' (/etc/haproxy/haproxy.cfg:50) tries t
o use incompatible tcp backend 'alertmanager' (/etc/haproxy/haproxy.cfg:173) in a 'use_backend' rule (see 'mode').
2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : http frontend 'omnibus-public-2-81' (/etc/haproxy/haproxy.cfg:50) tries t
o use incompatible tcp backend 'prometheus' (/etc/haproxy/haproxy.cfg:136) in a 'use_backend' rule (see 'mode').
2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : http frontend 'omnibus-public-2-81' (/etc/haproxy/haproxy.cfg:50) tries t
o use incompatible tcp backend 'metrics_public' (/etc/haproxy/haproxy.cfg:101) in a 'use_backend' rule (see 'mode').
2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : http frontend 'omnibus-public-2-81' (/etc/haproxy/haproxy.cfg:50) tries t
o use incompatible tcp backend 'metrics_public' (/etc/haproxy/haproxy.cfg:101) in a 'use_backend' rule (see 'mode').
2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : http frontend 'omnibus-public-2-81' (/etc/haproxy/haproxy.cfg:50) tries t
o use incompatible tcp backend 'metrics_public' (/etc/haproxy/haproxy.cfg:101) in a 'use_backend' rule (see 'mode').
2018-07-02 19:34:28 DEBUG reverseproxy-relation-changed [ALERT] 182/193428 (22324) : Fatal errors found in configuration.
2018-07-02 19:34:28 INFO juju-log reverseproxy:69: HAProxy configuration check failed, exiting.

This creates a weird serialization order dependence of configuration and relation. Configuration should be idempotent and resilient, and cleaning up these kinds of hook errors in production is fairly terrible with Juju 2.0. The hooks keep retrying and using resolved --no-retry will sometimes get "stuck" so mojo never sees a "steady state" to apply a manifest that might fix the issue.

Junien F (axino)
Changed in charm-haproxy:
status: New → Triaged
importance: Undecided → High
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.