Memcache connection limit too low

[James Dewey]

The memcache connection limit is left at the default (1024), this can cause issues if you are monitoring it (for instance by using a client to run a stats) in a relatively busy infrastructure.

I would suggest at the least parameterizing this in the playbooks, if not outright increasing the value.

[Kevin Carter (kevin-carter)]

Triaged, medium:

Template change in master https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/roles/memcached_server/templates/memcached.conf

New variable will need to be added IE:
 -c <num> max simultaneous connections (default: 1024)

[Kevin Carter (kevin-carter)]

Per: [ http://stackoverflow.com/questions/2054212/is-ulimit-needed-launching-memcached-with-c minor ] we will also need to update the /etc/security/limits.conf where memcached is installed such that the limit is set to the same value of the user defined connections when greater then 1024.

[Bjoern (bjoern-t)]

For a bigger deployment I do see around 400-500 connections which makes around 2k total connections for this particular deployment. Considering that we should expect that one host can always fail, I would set the default value to 2k, the configured memory is way big enough to support a magnitude of connections. I did create an rpc-maas issue to check memcache since a non working memcache can cause havoc in openstack. https://github.com/rcbops/rpc-maas/issues/176

[Bjoern (bjoern-t)]

Perhaps we should double the threads from 4 to 8 just in case we increase the connections again.

[Kevin Carter (kevin-carter)]

Doubling the threads is not needed. There is a default that can be set and should you find that you need more you can change that value from within your user_variables.yml file.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: kilo
Review: https://review.openstack.org/197729

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: juno
Review: https://review.openstack.org/197740

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/197729
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=ad4d1c313b599633102b90c58e16d35c15ae0a9d
Submitter: Jenkins
Branch: kilo

commit ad4d1c313b599633102b90c58e16d35c15ae0a9d
Author: Steve Lewis <email address hidden>
Date: Mon Jun 22 17:49:51 2015 -0700

    Configurable memcached connections limit & threads

    Increasing memcached_connections may require ulimit to be raised so
    that will be done if necessary.

    Increasing memcached_threads is not strongly recommended but is
    available if needed.

    Change-Id: I4b58d59f647e6f843fc33561dea75be67c0f78c4
    Closes-Bug: #1438498
    (cherry picked from commit d0127d51bdb9a6b51fec5d91f2dbd72177e4c4ca)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/197740
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=e9a0790b5a3f422616626ed1f59f8729a73c80ec
Submitter: Jenkins
Branch: juno

commit e9a0790b5a3f422616626ed1f59f8729a73c80ec
Author: Steve Lewis <email address hidden>
Date: Mon Jun 22 17:49:51 2015 -0700

    Configurable memcached connections limit & threads

    Increasing memcached_connections may require ulimit to be raised so
    that will be done if necessary.

    Increasing memcached_threads is not strongly recommended but is
    available if needed.

    Closes-Bug: #1438498
    (cherry picked from commit d0127d51bdb9a6b51fec5d91f2dbd72177e4c4ca)

    Conflicts:
     playbooks/roles/memcached_server/defaults/main.yml
     playbooks/roles/memcached_server/templates/memcached.conf
     rpc_deployment/roles/memcached/tasks/main.yml

    Change-Id: I6e624f49f2fe5c6865968a01927dfa9861c945ec

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.