Multiple hash index partitions causes overly large hash index

Bug #1018264 reported by Raghavendra D Prabhu on 2012-06-27
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Percona Server
George Ormond Lorch III
George Ormond Lorch III

Bug Description

Normally, adaptive hash index is initialized to 1/64 th of the buffer pool size.

in buf_pool_init:

 btr_search_sys_create(buf_pool_get_curr_size() / sizeof(void*) / 64);

However, innodb_adaptive_hash_index_partitions allows multiple adaptive hash index partitions to be created.

This leads to each partition being created with size of 1/64 of the buffer pool instead of 1/(64*number_of_partitions) and can cause overly high memory usage.

For instance, for a buffer pool of size 192G, without partitions it will be 3 GB but if the number of partitions is say, 8, it will be 24 G and will scale with that.

What is expected:
        The size of each partition should be decided after taking into account the number of buffer pool instances, ie.

in btr_search_sys_create:
                                    hash_size /= srv_buf_pool_instances

..otherwise each partition will be created with size of an otherwise single partition.

Related branches

Stewart Smith (community): Approve on 2012-09-28

On a second thought, instead of

                                     hash_size /= srv_buf_pool_instances

 hash_size /= btr_search_index_num may be better since that is more directly related to the adaptive hash index.

George Ormond Lorch III (gl-az) wrote :

5.1 does not support multiple adaptive hash search partitions so bug as described does not apply to 5.1

SingerWang (wang) wrote :

I think each partition being of 1/(64*number_of_partitions) make sense..

Yes, that seems to be the appropriate way to allocate starting partition
sizes. A fix for PS 5.5 has been made and is currently being tested.

On 9/26/2012 11:28 AM, SingerWang wrote:
> I think each partition being of 1/(64*number_of_partitions) make sense..

George O. Lorch III
Software Engineer, Percona Inc.
+1-888-401-3401 x542 US/Arizona (GMT -7)
skype: george.ormond.lorch.iii

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers