sometimes haproxy gets 504 when tries to reach keystone.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Mirantis OpenStack |
Fix Released
|
Critical
|
Alexander Makarov | ||
6.0.x |
Fix Released
|
High
|
Alexander Nevenchannyy |
Bug Description
[root@fuel ~]# fuel --fuel-version
api: '1.0'
astute_sha: f7cda2171b0b677
auth_required: true
build_id: 2015-02-23_20-49-44
build_number: '103'
feature_groups:
- mirantis
fuellib_sha: 3a441bce1b525fe
fuelmain_sha: a715aba8caf2003
nailgun_sha: adbc77b3435d148
ostf_sha: 3b57985d4d21555
production: docker
release: 6.0.1
release_versions:
2014.2-6.0.1:
VERSION:
api: '1.0'
astute_sha: f7cda2171b0b677
build_id: 2015-02-23_20-49-44
build_number: '103'
feature_
- mirantis
fuellib_sha: 3a441bce1b525fe
fuelmain_sha: a715aba8caf2003
nailgun_sha: adbc77b3435d148
ostf_sha: 3b57985d4d21555
production: docker
release: 6.0.1
Baremetal,Ubuntu, HA, Neutron-
Controllers:3 Computes:22
Deployment was successful, but durin rally tests I have found that keystone client (of rally) can't connect to server with 504 ERROR:
2015-02-25 07:36:49.925 19840 WARNING rally.common.broker [-] Failed to consume a task from the queue: Authorization Failed: Gateway Timeout (HTTP 504)
2015-02-25 07:36:49.928 19840 ERROR rally.common.broker [-] Gateway Timeout (HTTP 504)
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker Traceback (most recent call last):
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker consume(cache, queue.popleft())
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker tenant_id, user_dom)
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker user = self.client.
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker return self._create(
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker return self._post(url, body, response_key, return_raw, **kwargs)
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker resp, body = self.client.
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker return self.request(url, 'POST', **kwargs)
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker resp = super(LegacyJso
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker return self.session.
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker return func(*args, **kwargs)
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker File "/opt/stack/
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker raise exceptions.
2015-02-25 07:36:49.928 19840 TRACE rally.common.broker GatewayTimeout: Gateway Timeout (HTTP 504)
vip__public (ocf::mirantis:
vip__management (ocf::mirantis:
At the same time I can see in haproxy log I can see 504 response from keystone-server on primary controller:
<134>Feb 25 07:36:48 node-7 haproxy[14779]: 172.16.46.58:54712 [25/Feb/
keystone and haproxy logs from node-7 (primary controller) are attached
DG will be asap
Changed in mos: | |
assignee: | nobody → MOS Keystone (mos-keystone) |
Changed in mos: | |
status: | New → Confirmed |
importance: | Undecided → High |
milestone: | none → 6.1 |
Changed in mos: | |
assignee: | MOS Keystone (mos-keystone) → Alexander Makarov (amakarov) |
status: | Confirmed → In Progress |
Changed in mos: | |
status: | In Progress → Fix Committed |
tags: | added: customer-found support |
DG: https:/ /drive. google. com/a/mirantis. com/file/ d/0Bx4ptZV1Jt7h S2czNGxqMHdlT1k /view?usp= sharing