[cellv2] the performance issue of cellv2 when creating 500 instances concurrently
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Confirmed
|
Medium
|
Unassigned |
Bug Description
Description
===========
we test cellv2 solution and execute creating instances concurrently.
But there is the performance issue due to query instance info in scheduler process
Steps to reproduce
==================
use rally to execute creating 500 instances concurrently
Expected result
===============
all instances are created successfully
Actual result
=============
many instances failed due to timeout
Environment
===========
1. Exact version of OpenStack you are running. See the following
openstack-
openstack-
2. Which hypervisor did you use?
KVM. We have 2000 hosts
2. Which storage type did you use?
Our own storage device
3. Which networking type did you use?
Our own network device
Analysis
==============
In scheduler process, get_host_
_get_host_state call host_state.update and _get_instance_info
So _get_instance_info will be called at every request and every hosts.
Because cellv2 have disable instance info cache.
So there's a performance issue even we don't use affinity filter policy
Changed in nova: | |
assignee: | nobody → Surya Seetharaman (tssurya) |
tags: | added: cells scheduler |
Changed in nova: | |
assignee: | Matt Riedemann (mriedem) → Eric Fried (efried) |
Changed in nova: | |
assignee: | Eric Fried (efried) → Matt Riedemann (mriedem) |
Sorry for getting to this so long after it was opened. What do you mean by, "Because cellv2 have disable instance info cache."? I don't notice anything in the code that is deliberately disabling the instance info cache in the HostManager.