| 2014-10-02 23:27:16 |
Dave Chiluk |
bug |
|
|
added bug |
| 2014-10-03 00:53:46 |
Nobuto Murata |
bug |
|
|
added subscriber Nobuto MURATA |
| 2014-10-03 05:08:44 |
Dave Chiluk |
description |
As of commit 742ceaba530995da02ef5e5ac32f1478d60efd35 to the ubuntu-trusty kernel tree
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-trusty.git;a=commitdiff;h=742ceaba530995da02ef5e5ac32f1478d60efd35
Please see https://bugs.launchpad.net/linux/+bug/1328088 as well as it's use case directly applies to this tunable.
mhash_entries and mphash_entries are now a settable option for the kernel. These options adjust a hashtable that grows at n^2 the number of namespaces on the the machine. So if you have 4000 network namespaces on the neutron gateway 4000^2=16000000 entries will exist in the hashtable. Recently this value increased to 262144 entries by default *(4mb page size/sizeof(struct list_head)). That is enough to store 512 network namespaces without collisions in the hashtable.
So it is the recommendation of this engineer that deployments with large numbers of network namespaces add "mhash_entries=16000000 mphash_entries=16000000" to their kernel command line
This would be enough to fit 4000 namespaces without collission.
My suggestion is that this should be sold as an expected number of namespaces config option with a default value of 4000 for neutron nodes. The charm should then square the value and edit /etc/default/grub GRUB_CMDLINE_LINUX appropriately and update-grub. |
With the default sized hashtables, creation and deletion of instances in their own namespaces becomes prohibitively expensive. Without this change namespace creation and deletion can begin to take minutes instead of milliseconds when higher numbers of namespaces already exist on the machine (higher numbers = greater than 4000 namespaces).
As of commit 742ceaba530995da02ef5e5ac32f1478d60efd35 to the ubuntu-trusty kernel tree
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-trusty.git;a=commitdiff;h=742ceaba530995da02ef5e5ac32f1478d60efd35
Please see https://bugs.launchpad.net/linux/+bug/1328088 as well as it's use case directly applies to this tunable.
mhash_entries and mphash_entries are now a settable option for the kernel. These options adjust a hashtable that grows at n^2 the number of namespaces on the the machine. So if you have 4000 network namespaces on the neutron gateway 4000^2=16000000 entries will exist in the hashtable. Recently this value increased to 262144 entries by default *(4mb page size/sizeof(struct list_head)). That is enough to store 512 network namespaces without collisions in the hashtable.
So it is the recommendation of this engineer that deployments with large numbers of network namespaces add "mhash_entries=16000000 mphash_entries=16000000" to their kernel command line
This would be enough to fit 4000 namespaces without collission.
My suggestion is that this should be sold as an expected number of namespaces config option with a default value of 4000 for neutron nodes. The charm should then square the value and edit /etc/default/grub GRUB_CMDLINE_LINUX appropriately and update-grub. |
|
| 2014-10-03 05:09:37 |
Dave Chiluk |
description |
With the default sized hashtables, creation and deletion of instances in their own namespaces becomes prohibitively expensive. Without this change namespace creation and deletion can begin to take minutes instead of milliseconds when higher numbers of namespaces already exist on the machine (higher numbers = greater than 4000 namespaces).
As of commit 742ceaba530995da02ef5e5ac32f1478d60efd35 to the ubuntu-trusty kernel tree
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-trusty.git;a=commitdiff;h=742ceaba530995da02ef5e5ac32f1478d60efd35
Please see https://bugs.launchpad.net/linux/+bug/1328088 as well as it's use case directly applies to this tunable.
mhash_entries and mphash_entries are now a settable option for the kernel. These options adjust a hashtable that grows at n^2 the number of namespaces on the the machine. So if you have 4000 network namespaces on the neutron gateway 4000^2=16000000 entries will exist in the hashtable. Recently this value increased to 262144 entries by default *(4mb page size/sizeof(struct list_head)). That is enough to store 512 network namespaces without collisions in the hashtable.
So it is the recommendation of this engineer that deployments with large numbers of network namespaces add "mhash_entries=16000000 mphash_entries=16000000" to their kernel command line
This would be enough to fit 4000 namespaces without collission.
My suggestion is that this should be sold as an expected number of namespaces config option with a default value of 4000 for neutron nodes. The charm should then square the value and edit /etc/default/grub GRUB_CMDLINE_LINUX appropriately and update-grub. |
With the default sized hashtables, creation and deletion of instances in their own namespaces becomes prohibitively expensive. Without this change namespace creation and deletion can begin to take minutes instead of milliseconds when higher numbers of namespaces already exist on the machine (higher numbers = greater than 4000 namespaces).
As of commit 742ceaba530995da02ef5e5ac32f1478d60efd35 to the ubuntu-trusty kernel tree
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-trusty.git;a=commitdiff;h=742ceaba530995da02ef5e5ac32f1478d60efd35
Please see https://bugs.launchpad.net/linux/+bug/1328088 as well as it's use case directly applies to this tunable.
mhash_entries and mphash_entries are now a settable option for the kernel. These options adjust a hashtable that grows at n^2 the number of namespaces on the the machine. So if you have 4000 network namespaces on the neutron gateway 4000^2=16000000 entries will exist in the hashtable. Recently this value increased to 262144 entries by default *(4mb page size/sizeof(struct list_head)). That is enough to store 512 network namespaces without collisions in the hashtable.
So it is the recommendation of this engineer that deployments with large numbers of network namespaces add "mhash_entries=16000000 mphash_entries=16000000" to their kernel command line of at least the neutron gateway.
This would be enough to fit 4000 namespaces without collission.
My suggestion is that this should be sold as an expected number of namespaces config option with a default value of 4000 for neutron nodes. The charm should then square the value and edit /etc/default/grub GRUB_CMDLINE_LINUX appropriately and update-grub. |
|
| 2014-10-03 21:10:18 |
Jorge Niedbalski |
tags |
|
cts |
|
| 2014-10-06 13:04:53 |
Edward Hope-Morley |
tags |
cts |
cts openstack |
|
| 2014-11-24 09:50:55 |
James Page |
quantum-gateway (Juju Charms Collection): importance |
Undecided |
High |
|
| 2014-11-24 09:50:58 |
James Page |
quantum-gateway (Juju Charms Collection): status |
New |
Triaged |
|
| 2014-11-24 09:55:58 |
James Page |
quantum-gateway (Juju Charms Collection): importance |
High |
Medium |
|
| 2014-11-25 15:53:13 |
Ryan Beisner |
bug |
|
|
added subscriber Ryan Beisner |
| 2014-11-25 15:54:07 |
Ryan Beisner |
tags |
cts openstack |
cts openstack uosci |
|
| 2017-01-05 15:47:40 |
James Page |
affects |
quantum-gateway (Juju Charms Collection) |
neutron-gateway (Juju Charms Collection) |
|
| 2017-02-23 18:57:59 |
James Page |
charm-neutron-gateway: importance |
Undecided |
Medium |
|
| 2017-02-23 18:57:59 |
James Page |
charm-neutron-gateway: status |
New |
Triaged |
|
| 2017-02-23 18:58:01 |
James Page |
neutron-gateway (Juju Charms Collection): status |
Triaged |
Invalid |
|
