trove workers should use all available CPUs by default

Bug #1335284 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack DBaaS (Trove)
Fix Released
Medium
Matt Riedemann

Bug Description

Similar to nova commit 75c96a48fc7e5dfb59d8258142b01422f81b0253, glance bug 1333325 and cinder bug 1333370, trove has configuration values for the number of api and conductor workers it will run and currently defaults to 1. In a multiprocess host this is under-utilizing resources, so we should make a sane default of using the available CPUs for workers the default value.

Matt Riedemann (mriedem)
description: updated
Changed in trove:
assignee: nobody → Matt Riedemann (mriedem)
summary: - trove workers should just all available CPUs by default
+ trove workers should use all available CPUs by default
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to trove (master)

Fix proposed to branch: master
Review: https://review.openstack.org/103232

Changed in trove:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/103239

Changed in trove:
importance: Undecided → Medium
milestone: none → juno-2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to trove (master)
Download full text (4.1 KiB)

Reviewed: https://review.openstack.org/103232
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=f5b693983537d246a40b6837ecdfe616d89932f5
Submitter: Jenkins
Branch: master

commit f5b693983537d246a40b6837ecdfe616d89932f5
Author: Matt Riedemann <email address hidden>
Date: Fri Jun 27 12:55:42 2014 -0700

    Sync processutils from oslo with deps

    The main target for the sync is to pick up the get_worker_count()
    method in commit 85f178489a128a04a7ee3ed44018403caa109ef0 so that we
    can set trove api/conductor workers equal to the number of CPUs on the
    host.

    Changes:

    local
    -----
    12bcdb7 Remove vim header

    log
    ---
    109e325 Use oslo.messaging to publish log errors
    de4adbc pep8: fixed multiple violations
    eac71f5 Fix common.log.ContextFormatter for Python 3
    d78b633 Fixes a simple spelling mistake
    621d831 always log a traceback in the sys.excepthook
    90ae24b Remove redundant default=None for config options
    af36c2a Fix logging setup for Python 3.4
    cdcc19c Mask passwords that are included in commands
    fe3389e Improve help strings
    8a0f567 Remove str() from LOG.* and exceptions
    f61a4e7 Fix python26 compatibility for RFCSysLogHandler
    da6d713 Revert setting oslo-incubator logs to INFO
    0d18381 Set default log levels for oslo.messaging and oslo-incubator
    346884d Add default user_identity to logging record
    86707cd Remove None for dict.get()
    61ff7a6 Rename Openstack to OpenStack
    dda47c9 Use ContextFormatter for imparting context info
    52b6446 Fix deprecated messages sent multiple times
    1978114 default connectionpool to WARN log level
    79e8a9a Backport 'ident' from python 3.3 for Oslo's SysLogHandler
    55cd46a remove extra newlines that eventlet seems to add
    bcf6d5e Small edits on help strings
    d292f7d Add error type to unhandled exception log message
    0228b81 Logging excepthook: print exception info if debug=True
    71208fe Fix spelling errors in comments
    8b2b0b7 Use hacking import_exceptions for gettextutils._
    6d0a6c3 Correct invalid docstrings
    fe18bfa Remove keystone from default_log_levels default
    13eb01c Adding domain to context and log
    27b21bc Unify different names between Python2/3 with six.moves
    12bcdb7 Remove vim header
    a6f40ac Don't log to stdout when log_dir is set
    25c5854 Adds admin_password as key to be sanitized when logging
    2251cb5 Do not name variables as builtins
    cbfded9 Default iso8601 logging to WARN
    76b0cd1 Add mask password impl from other projects
    5fef5a8 allow keeping of existing loggers with fileConfig
    fb8075a Add amqp=WARN,qpid=WARN to default_log_levels
    4654a5d Don't override default value for eventlet.wsgi.server logging
    a817275 _get_log_file_path explictly return, when logfile/logdire unset
    e12e6b4 Make openstack.common.log Python 3 compatible
    572cfec Make Messages unicode before hitting logging
    36859e5 Adding instance_uuid to context and log

    processutils
    ------------
    85f1784 Move nova.utils.cpu_count() to processutils module
    cdcc19c Mask passwords that are included in commands
 ...

Read more...

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/103239
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=e932c4777d6cae1366c61c1becba1cbddf185378
Submitter: Jenkins
Branch: master

commit e932c4777d6cae1366c61c1becba1cbddf185378
Author: Matt Riedemann <email address hidden>
Date: Mon Jul 7 10:43:06 2014 -0700

    Use (# of CPUs) api/conductor workers by default

    This changes the default number of trove API and conductor workers to be
    equal to the number of CPUs available on the host, rather than
    defaulting to 1 as it did before.

    Commit 75c96a48fc7e5dfb59d8258142b01422f81b0253 did the same thing in
    Nova in Icehouse. Similar changes are being made to Glance and Cinder
    as well.

    DocImpact: trove_api_workeres and trove_conductor_workers will now be
               equal to the number of CPUs available by default if not
               explicitly specified in the trove configuration files.

    UpgradeImpact: Anyone upgrading to this change that does not have
               trove_api_workers or trove_conductor_workers specified in
               the trove configuration files will now be running multiple
               API and conductor workers by default when they restart the
               respective trove services.

    Closes-Bug: #1335284

    Change-Id: Id300bbe991436a0f826ea715630669ab5922a6a4

Changed in trove:
status: In Progress → Fix Committed
Changed in trove:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in trove:
milestone: juno-2 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.