setting rpc_conn_pool_size to 0 prevents processign of RPCs

Bug #1899533 reported by sean mooney
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.messaging
Fix Released
Undecided
sean mooney

Bug Description

during the root cause analasyis of downstream bug https://bugzilla.redhat.com/show_bug.cgi?id=1880371 it was observed that setting rpc_conn_pool_size to 0
prevents nova from processing rpc calls.

is defiend here to be any integer defaulting to 30
https://github.com/openstack/oslo.messaging/blob/e44c9883066d9b2d081a594b97aac3d598d491c9/oslo_messaging/_drivers/base.py#L25-L27

it should be defeiend as any positive integer with a min value of 1.

in the rabbit mq driver rpc_conn_pool_size is used to define the max size of the connection pool

https://github.com/openstack/oslo.messaging/blob/8d78ab2339d5798c7d8f5c49eaf6d2ef18859a07/oslo_messaging/_drivers/impl_rabbit.py#L1388

which is then used in put the get methods in the pool class.

https://github.com/openstack/oslo.messaging/blob/4fc9e70aa11f8cb0476a927ee471adbd296bba8f/oslo_messaging/_drivers/pool.py#L88-L105

when rpc_conn_pool_size==0 this loop never exits

to prevent this we shoudl add a min_size=1 to the rpc_conn_pool_size
config option.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (master)

Fix proposed to branch: master
Review: https://review.opendev.org/757654

Changed in oslo.messaging:
assignee: nobody → sean mooney (sean-k-mooney)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (master)

Reviewed: https://review.opendev.org/757654
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=00d15eaeaba0ded0330cdcec7b19eee3adbfb1e1
Submitter: Zuul
Branch: master

commit 00d15eaeaba0ded0330cdcec7b19eee3adbfb1e1
Author: Sean Mooney <email address hidden>
Date: Mon Oct 12 22:43:31 2020 +0100

    add min of 1 to rpc_conn_pool_size

    This change add a min value of 1 to
    [oslo_messaging_rabbit]/rpc_conn_pool_size
    such that there is always at least 1 connection avaiable.

    This change add a runtime check to ensure that
    [oslo_messaging_rabbit]/rpc_conn_pool_size is greater than
    or equal too [oslo_messaging_rabbit]/conn_pool_min_size

    Change-Id: I2ad4b9f1d012c9f0586a932ac27d96da1bcc4e4c
    Closes-Bug: #1899533

Changed in oslo.messaging:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/758414

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (stable/victoria)

Reviewed: https://review.opendev.org/758414
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=8f5847dcbe5144272ff7295fce2b50a9f544e5b0
Submitter: Zuul
Branch: stable/victoria

commit 8f5847dcbe5144272ff7295fce2b50a9f544e5b0
Author: Sean Mooney <email address hidden>
Date: Mon Oct 12 22:43:31 2020 +0100

    add min of 1 to rpc_conn_pool_size

    This change add a min value of 1 to
    [oslo_messaging_rabbit]/rpc_conn_pool_size
    such that there is always at least 1 connection avaiable.

    This change add a runtime check to ensure that
    [oslo_messaging_rabbit]/rpc_conn_pool_size is greater than
    or equal too [oslo_messaging_rabbit]/conn_pool_min_size

    Change-Id: I2ad4b9f1d012c9f0586a932ac27d96da1bcc4e4c
    Closes-Bug: #1899533
    (cherry picked from commit 00d15eaeaba0ded0330cdcec7b19eee3adbfb1e1)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/758793

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (stable/ussuri)

Reviewed: https://review.opendev.org/758793
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=7c9b070f5bc07692c837f9915eb0d8525065d0f6
Submitter: Zuul
Branch: stable/ussuri

commit 7c9b070f5bc07692c837f9915eb0d8525065d0f6
Author: Sean Mooney <email address hidden>
Date: Mon Oct 12 22:43:31 2020 +0100

    add min of 1 to rpc_conn_pool_size

    This change add a min value of 1 to
    [oslo_messaging_rabbit]/rpc_conn_pool_size
    such that there is always at least 1 connection avaiable.

    This change add a runtime check to ensure that
    [oslo_messaging_rabbit]/rpc_conn_pool_size is greater than
    or equal too [oslo_messaging_rabbit]/conn_pool_min_size

    Change-Id: I2ad4b9f1d012c9f0586a932ac27d96da1bcc4e4c
    Closes-Bug: #1899533
    (cherry picked from commit 00d15eaeaba0ded0330cdcec7b19eee3adbfb1e1)
    (cherry picked from commit 8f5847dcbe5144272ff7295fce2b50a9f544e5b0)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.messaging (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/759410

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.messaging (stable/train)

Reviewed: https://review.opendev.org/759410
Committed: https://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=425ef28924f9d4178eaea2c0c0fe9e580b796aaa
Submitter: Zuul
Branch: stable/train

commit 425ef28924f9d4178eaea2c0c0fe9e580b796aaa
Author: Sean Mooney <email address hidden>
Date: Mon Oct 12 22:43:31 2020 +0100

    add min of 1 to rpc_conn_pool_size

    This change add a min value of 1 to
    [oslo_messaging_rabbit]/rpc_conn_pool_size
    such that there is always at least 1 connection avaiable.

    This change add a runtime check to ensure that
    [oslo_messaging_rabbit]/rpc_conn_pool_size is greater than
    or equal too [oslo_messaging_rabbit]/conn_pool_min_size

    Modified:
      oslo_messaging/tests/functional/test_rabbitmq.py

    NOTE(stephenfin): We must set the 'conn_pool_min_size' config option in
    the 'RabbitMQFailoverTests.test_failover_scenario' test to avoid raising
    an error because the minimum is greater than the configured value. This
    wasn't necessary on stable/train due to the presence of change
    I0f86416623a0b718516147f0660b4df2b74cf867 ("Setup backend scenarios for
    functional tests") which changed how our functional tests ran.

    Change-Id: I2ad4b9f1d012c9f0586a932ac27d96da1bcc4e4c
    Closes-Bug: #1899533
    (cherry picked from commit 00d15eaeaba0ded0330cdcec7b19eee3adbfb1e1)
    (cherry picked from commit 8f5847dcbe5144272ff7295fce2b50a9f544e5b0)
    (cherry picked from commit 7c9b070f5bc07692c837f9915eb0d8525065d0f6)

tags: added: in-stable-train
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.