Add support for 'auto' number of API or conductor workers

[Russell Bryant]

Nova has some configuration options that allow you to have some services start multiple worker processes.

    [general]
    ec2_workers=
    osapi_compute_workers=
    metadata_workers=

    [conductor]
    workers=

Swift has a similar "workers" option. In Swift, you can set this option to 'auto', and it will use the number of CPU cores. We should add support for 'auto' to all of the workers options in Nova.

    https://git.openstack.org/cgit/openstack/swift/tree/etc/proxy-server.conf-sample

[Shantanu Tushar (shaan7in)]

I was looking for something to implement as my first patch and this sounds interesting. I did some poking around and found that nova uses Oslo.config which doesn't seem to support an `auto'-like feature. However, I do see this line[1] in nova/cmd/all.py -

    launcher.launch_service(server, workers=server.workers or 1)

which essentially starts worker processes. Does it make sense to detect the number of CPUs here and set that instead of 1?

[1] https://github.com/openstack/nova/blob/master/nova/cmd/all.py#L59

[Davanum Srinivas (DIMS) (dims-v)]

Shantanu,

Yes, You can borrow the snippet for CPU_COUNT from swift

https://git.openstack.org/cgit/openstack/swift/tree/swift/common/wsgi.py#n42

And then you can do

launcher.launch_service(server, workers=server.workers or CPU_COUNT)

instead of

launcher.launch_service(server, workers=server.workers or 1)

Good Luck with your first review :)

-- dims

[Shantanu Tushar (shaan7in)]

Davanum,

That is exactly what I am planning to do, just wanted a confirmation that it makes sense. Thanks :)

[Anton (agorenkov)]

As there is no visible activity on the issue during the last month, I'll try to look at it too.

[Shantanu Tushar (shaan7in)]

I have a patch at https://review.openstack.org/#/c/56820/1 which has a failed test due to some issue with test infra.

[Anton (agorenkov)]

Ok, so you are planning to finalize the fix, right?

[Shantanu Tushar (shaan7in)]

Yes, I am working on the related bug https://bugs.launchpad.net/openstack-ci/+bug/1252137

[Anton (agorenkov)]

Ok, so please assign the issues you are working on to yourself.

[Mohammed Naser (mnaser)]

This has been taken care of in this merged review

https://review.openstack.org/#/c/69266/