[Sahara] GET request with big cluster can take more than a minute

Bug #1545039 reported by Evgeny Sikachev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Invalid
High
Evgeny Sikachev
9.x
Invalid
High
Evgeny Sikachev

Bug Description

ENVIRONMENT: MOS 8.0 ISO 529(RC1)

If sahara has the big cluster, for example, CDH with 1 master, 1 manager and 300 workers, GET requests can take more than a minute. You can update timeout for connection to database in /etc/sahara/sahara.conf in section [connection]

This bug reproduced via UI and CLI sahara client

Changed in mos:
milestone: 8.0 → 8.0-updates
importance: Undecided → High
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

I suggest we take a look at this on Sahara side, maybe we are doing something very inefficiently.

But overall, with synchronous requests we are going to hit similar problems here and there as we scale. The fundamental solution should be using of pagination / asynchronous processing of requests / etc.

Vitaly, could you please elaborate what specific GET requests times out?

By the way, you would also need to tweak the timeout in haproxy.

Changed in mos:
status: New → Incomplete
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Oops, I meant Evgeny, sorry :(

Changed in mos:
assignee: MOS Sahara (mos-sahara) → Evgeny Sikachev (esikachev)
Revision history for this message
Evgeny Sikachev (esikachev) wrote :

Roman, this bug has been found only on 200 nodes scale and once. We don't know what happened, but this bug takes place

Revision history for this message
Evgeny Sikachev (esikachev) wrote :

move this bug to Invalid status, because it was reproduced only once. If it occurs again, a bug should be reopened.

Changed in mos:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.