Percona Server with XtraDB

Multiple hash index partitions causes overly large hash index

Reported by Raghavendra D Prabhu on 2012-06-27
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Percona Server
High
George Ormond Lorch III
5.1
High
Unassigned
5.5
High
George Ormond Lorch III

Bug Description

Normally, adaptive hash index is initialized to 1/64 th of the buffer pool size.

in buf_pool_init:

 btr_search_sys_create(buf_pool_get_curr_size() / sizeof(void*) / 64);

However, innodb_adaptive_hash_index_partitions allows multiple adaptive hash index partitions to be created.

This leads to each partition being created with size of 1/64 of the buffer pool instead of 1/(64*number_of_partitions) and can cause overly high memory usage.

For instance, for a buffer pool of size 192G, without partitions it will be 3 GB but if the number of partitions is say, 8, it will be 24 G and will scale with that.

What is expected:
        The size of each partition should be decided after taking into account the number of buffer pool instances, ie.

in btr_search_sys_create:
                                    hash_size /= srv_buf_pool_instances

..otherwise each partition will be created with size of an otherwise single partition.

Related branches

On a second thought, instead of

                                     hash_size /= srv_buf_pool_instances

 hash_size /= btr_search_index_num may be better since that is more directly related to the adaptive hash index.

George Ormond Lorch III (gl-az) wrote :

5.1 does not support multiple adaptive hash search partitions so bug as described does not apply to 5.1

SingerWang (wang) wrote :

I think each partition being of 1/(64*number_of_partitions) make sense..

Yes, that seems to be the appropriate way to allocate starting partition
sizes. A fix for PS 5.5 has been made and is currently being tested.

On 9/26/2012 11:28 AM, SingerWang wrote:
> I think each partition being of 1/(64*number_of_partitions) make sense..
>

--
George O. Lorch III
Software Engineer, Percona Inc.
+1-888-401-3401 x542 US/Arizona (GMT -7)
skype: george.ormond.lorch.iii

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers