_check_deploy_timeouts can attempt to spawn more processes then there are threads available

Bug #1285793 reported by Chris Krelle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ironic
Fix Released
Medium
Yuriy Zveryanskyy

Bug Description

in ironic/conductor/manager.py the function _check_deploy_timeouts could attempt to spawn more worker threads than there are available in the thread pool, resulting in spurious errors being logged (but not causing any service failure). This should be avoided by adding a query filter for provision_updated_at, sort oldest first, and use a reasonably small LIMIT (eg, smaller than rpc_thread_count)

This bug is based on review: https://review.openstack.org/#/c/72395

aeva black (tenbrae)
description: updated
Changed in ironic:
importance: Undecided → Medium
status: New → Triaged
Changed in ironic:
assignee: nobody → Yuriy Zveryanskyy (yzveryanskyy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/78651

Changed in ironic:
status: Triaged → In Progress
Revision history for this message
Openstack Gerrit (openstack-gerrit) wrote : Fix merged to ironic (master)

Reviewed: https://review.openstack.org/78651
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=d473bdabfdaf8fa2538de1056e804355a40979dc
Submitter: Jenkins
Branch: master

commit d473bdabfdaf8fa2538de1056e804355a40979dc
Author: Yuriy Zveryanskyy <email address hidden>
Date: Thu Mar 6 18:00:41 2014 +0200

    Add worker threads limit to _check_deploy_timeouts task

    New nodes filter parameters added to db.sqlalchemy.api. This
    parameters used in _check_deploy_timeouts periodic task for
    limit of worker threads which can be started simultaneously.
    New config option 'periodic_max_workers' added to conductor
    group.

    Closes-Bug: #1285793
    Change-Id: I646540e334dec05682640c05cedb315e0ee355bc

Changed in ironic:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in ironic:
milestone: none → juno-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in ironic:
milestone: juno-1 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.