Scheduler, performance impact when dealing with aggregates

[Sahid Orentino]

During a scheduling if we use filter that needs to get data from aggregates like CoreFilterAggregate, RamFilterAggregate does.
The filter retrieves metadata from the database for every host and can creates a performance impact if we have several hosts.

[John Garbutt (johngarbutt)]

Ideally lets fix this in a way that means that caching scheduler would cache this DB query.

So that probably means put it inside the host_manager, but see how that looks, it might be the wrong place/too deep

[Sylvain Bauza (sylvain-bauza)]

IMHO, we should at first make use of the conductor for placing a call to the DB.

On a long-term view, that's something possibly needing to be cached thanks to the no-db scheduler (even if it's not HostState).

[John Garbutt (johngarbutt)]

hmm, after the scheduler split, this should probably all come from NodeStats, add some location information in there... oh dear, I think we need the more complex fix here.

[John Garbutt (johngarbutt)]

So, the bigger fix, write aggregate info as a stat into the ComputeNode table, then use that to do the filtering.

This removes the DB query in the filter, and also ensures it works after the scheduler is removed from Nova.

[John Garbutt (johngarbutt)]

Hmm, maybe this should go through the blueprint process, to review the direction... Not 100% sure.

[Sylvain Bauza (sylvain-bauza)]

Well, as said in IRC, I agree that ResourceTracker should log in ComputeNode which aggregates the host is in.

[jiang, yunhong (yunhong-jiang)]

I also prefer to keep this in compute node, but I don't understand why it's a must to keep this in resource tracker after the split? I think even after the split, the scheduler may still need to support host aggregate, right?

Are there anyone working on a BP for it?

[wingwj (wingwj)]

I saw the discussion in Summit here https://etherpad.openstack.org/p/juno-nova-no-db-scheduler.

So if we add the compute cache in scheduler, is the issue solved?

[Sylvain Bauza (sylvain-bauza)]

Somewhat related to https://review.openstack.org/89893

[Sylvain Bauza (sylvain-bauza)]

As there will be work on how the Scheduler is getting updates from aggregates during the Kilo release, I'm holding this bug until the former is merged.

[Davanum Srinivas (DIMS) (dims-v)]

Sylvain, if you are picking this up, please re-assign it to yourself. thanks.

[Sahid Orentino (sahid-ferdjaoui)]

We can probably close that one

  https://review.openstack.org/#/c/159904/

[Sylvain Bauza (sylvain-bauza)]

I would rather mark it as invalid now that the design changed.

I really would like to keep fix committed to match with Gerrit changes

[Sahid Orentino (sahid-ferdjaoui)]

Invalid is not the good status IMHO - Probably make more sense to affiliate it to the blueprint related since this bug was the start of thinking the problem, then won't fix or fixed seem to be better I mean for an history point of view. But anyway...