reverseproxy-relation-changed fails configuration check failed

Bug #1520305 reported by mahmoh
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
haproxy (Juju Charms Collection)
New
Undecided
Unassigned

Bug Description

haproxy unknown false cs:trusty/haproxy-10

unit-haproxy0-0[905]: 2015-11-26 16:20:12 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 #015Reading package lists... 0%#015#015Reading package lists... 0%#015#015Reading package lists... 1%#015#015Reading package lists... 10%#015#015Reading package lists... 10%#015#015Reading package lists... 10%#015#015Reading package lists... 10%#015#015Reading package lists... 18%#015#015Reading package lists... 33%#015#015Reading package lists... 44%#015#015Reading package lists... 48%#015#015Reading package lists... 48%#015#015Reading package lists... 49%#015#015Reading package lists... 49%#015#015Reading package lists... 54%#015#015Reading package lists... 54%#015#015Reading package lists... 55%#015#015Reading package lists... 55%#015#015Reading package lists... 55%#015#015Reading package lists... 55%#015#015Reading package lists... 63%#015#015Reading package lists... 78%#015#015Reading package lists... 78%#015#015Reading package lists... 83%#015#015Reading package lists... 83%#015#015Reading package lists... 83%#015#015Reading package lists... 83%#015#015Reading package lists... 84%#015#015Reading package lists... 85%#015#015Reading package lists... 85%#015#015Reading package lists... 86%#015#015Reading package lists... 86%#015#015Reading package lists... 90%#015#015Reading package lists... 90%#015#015Reading package lists... 90%#015#015Reading package lists... 90%#015#015Reading package lists... 90%#015#015Reading package lists... 90%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 91%#015#015Reading package lists... 94%#015#015Reading package lists... 94%#015#015Reading package lists... 95%#015#015Reading package lists... 95%#015#015Reading package lists... 99%#015#015Reading package lists... 99%#015#015Reading package lists... 99%#015#015Reading package lists... 99%#015#015Reading package lists... Done
unit-haproxy0-0[905]: 2015-11-26 16:20:12 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 #015Building dependency tree... 0%#015#015Building dependency tree... 0%#015#015Building dependency tree... 50%#015#015Building dependency tree... 50%#015#015Building dependency tree... 67%#015#015Building dependency tree
unit-haproxy0-0[905]: 2015-11-26 16:20:12 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 #015Reading state information... 0%#015#015Reading state information... 4%#015#015Reading state information... Done
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.juju-log server.go:254 reverseproxy:12: Unit 'landscape-server/0' overrides 'services', skipping further processing.
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.juju-log server.go:254 reverseproxy:12: Service: haproxy_service
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.juju-log server.go:254 reverseproxy:12: Service: landscape-https
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.juju-log server.go:254 reverseproxy:12: Service: landscape-http
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.juju-log server.go:254 reverseproxy:12: enable_monitoring must be set to true for metrics
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 [ALERT] 329/162014 (2106) : parsing [/etc/haproxy/haproxy.cfg:26] : 'redirect' expects 'code', 'prefix', 'location', 'set-cookie', 'clear-cookie', 'drop-query' or 'append-slash' (was 'scheme').
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 [ALERT] 329/162014 (2106) : parsing [/etc/haproxy/haproxy.cfg:56] : 'bind' only supports the 'transparent', 'defer-accept', 'name', 'id', 'mss' and 'interface' options.
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 [ALERT] 329/162014 (2106) : parsing [/etc/haproxy/haproxy.cfg:60]: unknown parameter 'X-Forwarded-Proto', expects 'allow', 'deny', 'auth'.
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.reverseproxy-relation-changed logger.go:40 [ALERT] 329/162014 (2106) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
unit-haproxy0-0[905]: 2015-11-26 16:20:14 INFO unit.haproxy0/0.juju-log server.go:254 reverseproxy:12: HAProxy configuration check failed, exiting.
unit-haproxy0-0[905]: 2015-11-26 16:20:14 ERROR juju.worker.uniter.operation runhook.go:103 hook "reverseproxy-relation-changed" failed: exit status 1

Broken:

ii haproxy 1.4.24-2ubuntu0.2 amd64 fast and reliable load balancing reverse proxy

Working:

ii haproxy 1.5.4-1ubuntu2.1~ubuntu14.04.1 amd64 fast and reliable load balancing reverse proxy

$ juju version
1.24.7-trusty-amd64

$ diff /tmp/haproxy.cfg-broken /tmp/haproxy.cfg-working
2,7c2,8
< log /dev/log local0
< log /dev/log local1 notice
< chroot /var/lib/haproxy
< user haproxy
< group haproxy
< daemon
---
> log 127.0.0.1 local0
> log 127.0.0.1 local1 notice
> maxconn 4096
> user haproxy
> group haproxy
> spread-checks 0
> tune.ssl.default-dh-param 1024
10,23c11,149
< log global
< mode http
< option httplog
< option dontlognull
< contimeout 5000
< clitimeout 50000
< srvtimeout 50000
< errorfile 400 /etc/haproxy/errors/400.http
< errorfile 403 /etc/haproxy/errors/403.http
< errorfile 408 /etc/haproxy/errors/408.http
< errorfile 500 /etc/haproxy/errors/500.http
< errorfile 502 /etc/haproxy/errors/502.http
< errorfile 503 /etc/haproxy/errors/503.http
< errorfile 504 /etc/haproxy/errors/504.http
---
> log global
> mode http
> option httplog
> option dontlognull
> retries 3
> timeout queue 60000
> timeout connect 5000
> timeout client 120000
> timeout server 120000
>
> frontend haproxy-0-10000
> bind 0.0.0.0:10000
> default_backend haproxy_monitoring
> mode http
> acl allowed_cidr src 0.0.0.0/0
> http-request deny unless allowed_cidr
>
> backend haproxy_monitoring
> mode http
> stats enable
> stats uri /
> stats realm Haproxy\ Statistics
> stats auth haproxy:haproxy
> stats refresh 3
>
> frontend haproxy-0-80
> bind 0.0.0.0:80
> default_backend landscape-http
> mode http
> timeout client 300000
> acl ping path_beg -i /ping
> redirect scheme https unless ping
> use_backend landscape-ping if ping
>
> backend landscape-http
> mode http
> timeout server 300000
> balance leastconn
> option httpchk HEAD / HTTP/1.0
> errorfile 403 /var/lib/haproxy/service_landscape-http/403.http
> errorfile 500 /var/lib/haproxy/service_landscape-http/500.http
> errorfile 502 /var/lib/haproxy/service_landscape-http/502.http
> errorfile 503 /var/lib/haproxy/service_landscape-http/503.http
> errorfile 504 /var/lib/haproxy/service_landscape-http/504.http
> server landscape-appserver-landscape-server-0-0 172.16.9.116:8080 check inter 5000 rise 2 fall 5 maxconn 50
> server landscape-appserver-landscape-server-0-1 172.16.9.116:8081 check inter 5000 rise 2 fall 5 maxconn 50
>
> backend landscape-ping
> mode http
> timeout server 300000
> balance leastconn
> option httpchk HEAD / HTTP/1.0
> errorfile 403 /var/lib/haproxy/service_landscape-http/403.http
> errorfile 500 /var/lib/haproxy/service_landscape-http/500.http
> errorfile 502 /var/lib/haproxy/service_landscape-http/502.http
> errorfile 503 /var/lib/haproxy/service_landscape-http/503.http
> errorfile 504 /var/lib/haproxy/service_landscape-http/504.http
> server landscape-pingserver-landscape-server-0-0 172.16.9.116:8070 check inter 5000 rise 2 fall 5 maxconn 50
> server landscape-pingserver-landscape-server-0-1 172.16.9.116:8071 check inter 5000 rise 2 fall 5 maxconn 50
>
> frontend haproxy-0-443
> bind 0.0.0.0:443 ssl crt /var/lib/haproxy/default.pem
> default_backend landscape-https
> mode http
> timeout client 300000
> http-request set-header X-Forwarded-Proto https
> acl message path_beg -i /message-system
> acl attachment path_beg -i /attachment
> acl api path_beg -i /api
> use_backend landscape-message if message
> use_backend landscape-message if attachment
> use_backend landscape-api if api
> acl package-upload path_beg -i /upload
> use_backend landscape-package-upload if package-upload
>
> backend landscape-https
> mode http
> timeout server 300000
> balance leastconn
> option httpchk HEAD / HTTP/1.0
> reqrep ^([^\ ]*)\ /upload/(.*) \1\ /\2
> errorfile 403 /var/lib/haproxy/service_landscape-https/403.http
> errorfile 500 /var/lib/haproxy/service_landscape-https/500.http
> errorfile 502 /var/lib/haproxy/service_landscape-https/502.http
> errorfile 503 /var/lib/haproxy/service_landscape-https/503.http
> errorfile 504 /var/lib/haproxy/service_landscape-https/504.http
> server landscape-appserver-landscape-server-0-0 172.16.9.116:8080 check inter 5000 rise 2 fall 5 maxconn 50
> server landscape-appserver-landscape-server-0-1 172.16.9.116:8081 check inter 5000 rise 2 fall 5 maxconn 50
>
> backend landscape-message
> mode http
> timeout server 300000
> balance leastconn
> option httpchk HEAD / HTTP/1.0
> reqrep ^([^\ ]*)\ /upload/(.*) \1\ /\2
> errorfile 403 /var/lib/haproxy/service_landscape-https/403.http
> errorfile 500 /var/lib/haproxy/service_landscape-https/500.http
> errorfile 502 /var/lib/haproxy/service_landscape-https/502.http
> errorfile 503 /var/lib/haproxy/service_landscape-https/503.http
> errorfile 504 /var/lib/haproxy/service_landscape-https/504.http
> server landscape-message-server-landscape-server-0-0 172.16.9.116:8090 check inter 5000 rise 2 fall 5 maxconn 50
> server landscape-message-server-landscape-server-0-1 172.16.9.116:8091 check inter 5000 rise 2 fall 5 maxconn 50
>
> backend landscape-api
> mode http
> timeout server 300000
> balance leastconn
> option httpchk HEAD / HTTP/1.0
> reqrep ^([^\ ]*)\ /upload/(.*) \1\ /\2
> errorfile 403 /var/lib/haproxy/service_landscape-https/403.http
> errorfile 500 /var/lib/haproxy/service_landscape-https/500.http
> errorfile 502 /var/lib/haproxy/service_landscape-https/502.http
> errorfile 503 /var/lib/haproxy/service_landscape-https/503.http
> errorfile 504 /var/lib/haproxy/service_landscape-https/504.http
> server landscape-api-landscape-server-0 172.16.9.116:9080 check inter 5000 rise 2 fall 5 maxconn 50
>
> backend landscape-package-upload
> mode http
> timeout server 300000
> balance leastconn
> option httpchk HEAD / HTTP/1.0
> reqrep ^([^\ ]*)\ /upload/(.*) \1\ /\2
> errorfile 403 /var/lib/haproxy/service_landscape-https/403.http
> errorfile 500 /var/lib/haproxy/service_landscape-https/500.http
> errorfile 502 /var/lib/haproxy/service_landscape-https/502.http
> errorfile 503 /var/lib/haproxy/service_landscape-https/503.http
> errorfile 504 /var/lib/haproxy/service_landscape-https/504.http
> server landscape-package-upload-landscape-server-0 172.16.9.116:9100 check inter 5000 rise 2 fall 5 maxconn 50
>
> frontend haproxy-0-80
> bind 0.0.0.0:80
> default_backend haproxy_service
>
> backend haproxy_service
> balance leastconn
> cookie SRVNAME insert
>
>
>

Revision history for this message
mahmoh (mahmoh) wrote :
Download full text (13.2 KiB)

...
unit-landscape-server-0: 05:35:35 INFO unit.landscape-server/0.website-relation-changed ...done.
unit-landscape-server-0: 05:35:35 INFO unit.landscape-server/0.website-relation-changed * Restarting Landscape standalone appserver 2
unit-haproxy-0: 05:35:40 DEBUG juju.worker.uniter.remotestate got unit change
unit-haproxy-0: 05:35:40 INFO juju.worker.uniter awaiting error resolution for "relation-changed" hook
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter.operation running operation skip run relation-changed (3; landscape-server/0) hook
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter.operation preparing operation "skip run relation-changed (3; landscape-server/0) hook"
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter.operation committing operation "skip run relation-changed (3; landscape-server/0) hook"
unit-haproxy-0: 05:35:41 INFO juju.worker.uniter no operations in progress; waiting for changes
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter.operation running operation run relation-changed (3; landscape-server/0) hook
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter acquire lock "machine-lock" for uniter hook execution
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter lock "machine-lock" acquired
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter.operation preparing operation "run relation-changed (3; landscape-server/0) hook"
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter.operation executing operation "run relation-changed (3; landscape-server/0) hook"
unit-haproxy-0: 05:35:41 DEBUG juju.worker.uniter [AGENT-STATUS] executing: running reverseproxy-relation-changed hook
unit-haproxy-0: 05:35:42 DEBUG juju.worker.leadership haproxy/0 renewing lease for haproxy leadership
unit-haproxy-0: 05:35:42 DEBUG juju.worker.leadership checking haproxy/0 for haproxy leadership

unit-haproxy-0: 05:35:44 DEBUG juju.worker.leadership haproxy/0 confirmed for haproxy leadership until 2016-10-20 12:36:42.808751163 +0000 UTC
unit-haproxy-0: 05:35:44 INFO juju.worker.leadership haproxy/0 will renew haproxy leadership at 2016-10-20 12:36:12.808751163 +0000 UTC
unit-haproxy-0: 05:35:44 DEBUG worker.uniter.jujuc hook context id "haproxy/0-reverseproxy-relation-changed-5462992257327414466"; dir "/var/lib/juju/agents/unit-haproxy-0/charm"
unit-haproxy-0: 05:35:45 DEBUG juju.worker.uniter.remotestate got unit change
Reading package lists... Done unit.haproxy/0.reverseproxy-relation-changed
Building dependency tree nit.haproxy/0.reverseproxy-relation-changed
Reading state information... Donet.haproxy/0.reverseproxy-relation-changed
unit-haproxy-0: 05:35:49 DEBUG worker.uniter.jujuc hook context id "haproxy/0-reverseproxy-relation-changed-5462992257327414466"; dir "/var/lib/juju/agents/unit-haproxy-0/charm"
unit-haproxy-0: 05:35:49 DEBUG worker.uniter.jujuc hook context id "haproxy/0-reverseproxy-relation-changed-5462992257327414466"; dir "/var/lib/juju/agents/unit-haproxy-0/charm"
unit-haproxy-0: 05:35:49 DEBUG worker.uniter.jujuc hook context id "haproxy/0-reverseproxy-relation-changed-5462992257327414466"; dir "/var/lib/juju/agents/unit-haproxy-0/charm"
unit-haproxy-0: 05:35:49 DEBUG worker.uniter.jujuc hook context id "haproxy/0-reversep...

Revision history for this message
mahmoh (mahmoh) wrote :
Revision history for this message
mahmoh (mahmoh) wrote :
Revision history for this message
mahmoh (mahmoh) wrote :

2.0.0-xenial-amd64

2.0.0+bzr5189-0ubuntu1~16.04.1

Deploying to KVM VM with LXD containers, also tried haproxy on the "metal" (KVM)

Revision history for this message
mahmoh (mahmoh) wrote :

juju deploy cs:bundle/landscape-dense-maas-1

^deployed via the gui

Revision history for this message
mahmoh (mahmoh) wrote :
Revision history for this message
mahmoh (mahmoh) wrote :

$ juju status
Model Controller Cloud/Region Version
landscape cavmaas cavmaas 2.0.0

App Version Status Scale Charm Store Rev OS Notes
haproxy error 1 haproxy jujucharms 16 ubuntu exposed
haproxy-a error 1 haproxy jujucharms 37 ubuntu exposed
landscape-server active 1 landscape-server jujucharms 15 ubuntu
postgresql active 1 postgresql jujucharms 40 ubuntu
rabbitmq-server active 1 rabbitmq-server jujucharms 43 ubuntu

Unit Workload Agent Machine Public address Ports Message
haproxy-a/0* error idle 0 172.16.3.2 hook failed: "reverseproxy-relation-changed" for landscape-server:website
haproxy/1* error idle 1 172.16.3.7 hook failed: "reverseproxy-relation-changed" for landscape-server:website
landscape-server/0* active idle 0/lxd/1 172.16.3.4
postgresql/0* active idle 0/lxd/3 172.16.3.6 5432/tcp Live master
rabbitmq-server/0* active idle 0/lxd/2 172.16.3.5 5672/tcp Unit is ready

Machine State DNS Inst id Series AZ
0 started 172.16.3.2 4y3h7x xenial default
0/lxd/1 started 172.16.3.4 juju-53e854-0-lxd-1 trusty
0/lxd/2 started 172.16.3.5 juju-53e854-0-lxd-2 trusty
0/lxd/3 started 172.16.3.6 juju-53e854-0-lxd-3 trusty
1 started 172.16.3.7 4y3h83 trusty default

Relation Provides Consumes Type
peer haproxy haproxy peer
website haproxy landscape-server regular
peer haproxy-a haproxy-a peer
website haproxy-a landscape-server regular
db-admin landscape-server postgresql regular
amqp landscape-server rabbitmq-server regular
replication postgresql postgresql peer
cluster rabbitmq-server rabbitmq-server peer

Revision history for this message
Francis Ginther (fginther) wrote :

The comments from mahmoh are the result of a different bug: lp:1635340.

Revision history for this message
Riccardo Magrini (riccardo-magrini) wrote :

the same issue during the deploy of landscape via MAAS Version 2.1.5, JUJU version 2.1.2-xenial-amd64 on Ubuntu 16.04.2 LTS Server.

The following juju status:

$:juju status
Model Controller Cloud/Region Version
landscapelab maaslab-controller maaslab 2.0.2

App Version Status Scale Charm Store Rev OS Notes
haproxy error 1 haproxy jujucharms 41 ubuntu exposed
landscape-server unknown 1 landscape-server jujucharms 21 ubuntu
postgresql active 1 postgresql jujucharms 101 ubuntu
rabbitmq-server active 1 rabbitmq-server jujucharms 5 ubuntu

Unit Workload Agent Machine Public address Ports Message
haproxy/2* error idle 2/lxd/2 10.20.81.5 hook failed: "reverseproxy-relation-changed" for landscape-server:website
landscape-server/2* unknown idle 2/lxd/0 10.20.81.3
postgresql/2* active idle 2/lxd/1 10.20.81.4 5432/tcp Live master (9.5.6)
rabbitmq-server/2* active idle 2/lxd/3 10.20.81.6 5672/tcp Unit is ready

Machine State DNS Inst id Series AZ
2 started 10.20.81.2 t3xfkw xenial default
2/lxd/0 started 10.20.81.3 juju-37fbed-2-lxd-0 xenial
2/lxd/1 started 10.20.81.4 juju-37fbed-2-lxd-1 xenial
2/lxd/2 started 10.20.81.5 juju-37fbed-2-lxd-2 xenial
2/lxd/3 started 10.20.81.6 juju-37fbed-2-lxd-3 xenial

Relation Provides Consumes Type
peer haproxy haproxy peer
website haproxy landscape-server regular
db-admin landscape-server postgresql regular
amqp landscape-server rabbitmq-server regular
replication postgresql postgresql peer
cluster rabbitmq-server rabbitmq-server peer

Revision history for this message
Riccardo Magrini (riccardo-magrini) wrote :

update on my lab:

during some tests, I've tried to deploy that bundle using this time the cli with the following command:

$: juju deploy landscape-dense-maas

as the right procedure, the node is passed in deployed status and juju has deployed the bundle without receive any issue... re-build the lab but using the cli or the gui with the command:

$: juju deploy cs:bundle/landscape-dense-maas-7

in both the case the issue with HAproxy is always present.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.