ceph-rgw is listening on all addresses

Bug #1596229 reported by Dave Walker
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Fix Released
High
Dave Walker

Bug Description

In stable/mitaka, it was possible to co-locate compute and storage on the same host. This now isn't possible because ceph-rgw is listening on 0.0.0.0:6780

However, on trunk (newton) this fails with:
fatal: [HOST]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "haproxy", "/usr/local/bin/kolla_ensure_haproxy_latest_config"], "delta": "0:00:00.019332", "end": "2016-06-25 17:12:05.012291", "failed": true, "rc": 1, "start": "2016-06-25 17:12:04.992959", "stderr": "Error response from daemon: Container c2d3ea84088f820433cb09df4fdfbf119dba91b6ba1fd96143fbb2fd64634b5e is restarting, wait until the container is running", "stdout": "", "stdout_lines": [], "warnings": []}

The container is in a restart loop with:
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Removing existing destination: /etc/haproxy/haproxy.cfg
INFO:__main__:Copying /var/lib/kolla/config_files/haproxy.cfg to /etc/haproxy/haproxy.cfg
INFO:__main__:Setting permissions for /etc/haproxy/haproxy.cfg
INFO:__main__:/var/lib/kolla/config_files/haproxy.pem does not exist, but is not required
INFO:__main__:Writing out command to execute
Running command: '/usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid'
<7>haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
[WARNING] 176/171528 (9) : config : 'option forwardfor' ignored for proxy 'mariadb' as it requires HTTP mode.
[ALERT] 176/171528 (9) : Starting proxy radosgw: cannot bind socket [10.131.161.101:6780]
<5>haproxy-systemd-wrapper: exit, haproxy RC=256

(10.131.161.101 is the VIP)

ceph-rgw is already listening on:
tcp 0 0 0.0.0.0:6780 0.0.0.0:* LISTEN -

This means that haproxy cannot listen on {VIP}:6780

Dave Walker (davewalker)
summary: - cannot co-locate compute and storage on trunk
+ ceph-rgw is listening on all addresses
Revision history for this message
Dave Walker (davewalker) wrote :
Changed in kolla:
status: New → In Progress
assignee: nobody → Dave Walker (davewalker)
Revision history for this message
Dave Walker (davewalker) wrote :

This was introduced by a fix for bug 1576817 (which I cannot reproduce)

Changed in kolla:
importance: Undecided → High
milestone: none → newton-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/334202
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=d5d04587f12e6632df5879b33adf1666914e1764
Submitter: Jenkins
Branch: master

commit d5d04587f12e6632df5879b33adf1666914e1764
Author: Dave Walker (Daviey) <email address hidden>
Date: Sat Jun 25 22:59:46 2016 +0100

    Revert "Fix radosgw 'rgw frontends' param"

    The original commit changes ceph-rgw to listen on all
    addresses (0.0.0.0:6780), which means that it listens on
    both the storage_interface address and the VIP.

    This makes it incompatible with having HAProxy on the same
    host.

    This reverts commit 819ae24b50ffda74e5b3a72f4c26321e7c9696cb.

    Change-Id: I6cba31cc942a0be9f65d2f4847c8c2bc364a9c1e
    Closes-Bug: 1596229
    Signed-off-by: Dave Walker (Daviey) <email address hidden>

Changed in kolla:
status: In Progress → Fix Released
Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/kolla 3.0.0.0b2

This issue was fixed in the openstack/kolla 3.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.