Comment 4 for bug 1597451

Revision history for this message
Elena Ezhova (eezhova) wrote :

@Michael, it's great that you are working on a fix, but the problem with 'dns-integration' extension is apparently not the only issue we are facing. With your patch failover fails on listeners update with the following 500 error in health manager logs: http://paste.openstack.org/show/526520/ (I only included a small part of the log, can provide the full snippet if needed).

And on the failover amphora side there are the following errors:

amphora-agent.log
2016-07-06 08:26:54.871 1339 INFO werkzeug [-] 192.168.0.9 - - [06/Jul/2016 08:26:54] "PUT /0.5/listeners/b8ba451c-3ff9-40a9-ab2c-86ecba8cdfeb/1047ff69-86dc-46fc-b295-8222c89c0d2b/haproxy HTTP/1.1" 202 -
2016-07-06 08:26:54.922 1339 INFO werkzeug [-] 192.168.0.9 - - [06/Jul/2016 08:26:54] "GET /0.5/listeners/1047ff69-86dc-46fc-b295-8222c89c0d2b HTTP/1.1" 200 -
2016-07-06 08:26:55.203 1339 DEBUG octavia.amphorae.backends.agent.api_server.listener [-] Failed to start HAProxy service: Command '['/usr/sbin/service', 'haproxy-1047ff69-86dc-46fc-b295-8222c89c0d2b', 'start']' returned non-zero exit status 1 start_stop_listener /usr/local/lib/python2.7/dist-packages/octavia/amphorae/backends/agent/api_server/listener.py:187
2016-07-06 08:26:55.239 1339 INFO werkzeug [-] 192.168.0.9 - - [06/Jul/2016 08:26:55] "PUT /0.5/listeners/1047ff69-86dc-46fc-b295-8222c89c0d2b/start HTTP/1.1" 500 -

ubuntu@amphora-b8ba451c-3ff9-40a9-ab2c-86ecba8cdfeb:~$ sudo cat /var/log/upstart/haproxy-1047ff69-86dc-46fc-b295-8222c89c0d2b.log
sort: cannot read: /var/lib/octavia/plugged_interfaces: No such file or directory

If I try to trigger a failover in case affected load balancer doesn't have a listener, error spring out all over octavia, nova-compute and neutron-server.
I should probably have had to devote more time to finding the root cause, but I didn't want to upload an incomplete fix.