innodb_dict_size_limit tries to do LRU eviction of an index that is in use
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Percona Server moved to https://jira.percona.com/projects/PS |
Invalid
|
Undecided
|
Unassigned | ||
5.1 |
Fix Released
|
High
|
George Ormond Lorch III | ||
5.5 |
Fix Released
|
High
|
George Ormond Lorch III | ||
5.6 |
Invalid
|
Undecided
|
Unassigned |
Bug Description
The Percona Server version is 5.5.33-31.1.
The server crashed while doing an UNDO of a long running UPDATE query that had modified thousands of rows. The assertion raised was as follows:
131110 14:09:59 InnoDB: Assertion failure in thread 16 in file sync0rw.c line 319
InnoDB: Failing assertion: lock->lock_word == X_LOCK_DECR
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://
InnoDB: about forcing recovery.
14:09:59 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://
key_buffer_
read_buffer_
max_used_
max_threads=5002
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
/mysql/
/mysql/
/lib/amd64/
/lib/amd64/
/lib/amd64/
/lib/amd64/
/lib/amd64/
/mysql/
/mysql/
/mysql/
/mysql/
/mysql/
/mysql/
/mysql/
/mysql/
/mysql/
/mysql/
/lib/amd64/
/lib/amd64/
Please read http://
and follow instructions on how to resolve the stack trace.
Resolved stack trace is much more helpful in diagnosing the
problem, so please do resolve it
You may download the Percona Server operations manual by visiting
http://
in the manual which will help you identify the cause of the crash.
And the resolved stack trace is as follows:
af909b rw_lock_free_func + 411
b66110 dict_index_
b66230 dict_table_
b6655d dict_table_LRU_trim + 269
b666f9 dict_table_
bf863f row_undo_mod + 95
bf02b8 row_undo_step + 104
be5638 que_run_threads + 1400
b06a9b trx_rollback_
b070cf trx_rollback_
This is how data dictionary has been limited:
innodb_
Related branches
- Laurynas Biveinis (community): Approve
- Vlad Lesin (community): Needs Fixing (g2)
-
Diff: 288 lines (+156/-32)8 files modifiedPercona-Server/mysql-test/suite/innodb_plugin/r/percona_bug758788.result (+9/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_bug758788-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_bug758788.test (+24/-0)
Percona-Server/storage/innodb_plugin/dict/dict0dict.c (+95/-32)
Percona-Server/storage/innodb_plugin/dict/dict0mem.c (+1/-0)
Percona-Server/storage/innodb_plugin/include/lock0lock.h (+10/-0)
Percona-Server/storage/innodb_plugin/include/lock0lock.ic (+14/-0)
Percona-Server/storage/innodb_plugin/lock/lock0lock.c (+2/-0)
- Laurynas Biveinis (community): Approve
- Vlad Lesin (community): Needs Fixing (g2)
-
Diff: 288 lines (+156/-32)8 files modifiedmysql-test/suite/innodb/r/percona_bug758788.result (+9/-0)
mysql-test/suite/innodb/t/percona_bug758788-master.opt (+1/-0)
mysql-test/suite/innodb/t/percona_bug758788.test (+24/-0)
storage/innobase/dict/dict0dict.c (+95/-32)
storage/innobase/dict/dict0mem.c (+1/-0)
storage/innobase/include/lock0lock.h (+10/-0)
storage/innobase/include/lock0lock.ic (+14/-0)
storage/innobase/lock/lock0lock.c (+2/-0)
summary: |
- innodb_dict_size_limit tries to do a LRU of an index that is in use + innodb_dict_size_limit tries to do LRU eviction of an index that is in + use |
tags: | added: xtradb |
I also find the below semaphore waits possibly related to LRU eviction of index.
InnoDB: Warning: a long semaphore wait:
--Thread 22 has waited at row0purge.c line 682 for 241.00 seconds the semaphore:
Mutex at 27294d8a8 '&dict_sys->mutex', lock var 1
waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: Pending preads 0, pwrites 0
======= ======= ======= ======= ======= == ======= ======= ======= ======= == latch_part[ i]' tmp.a5aenR/ Percona- Server- 5.5.33- rel31.1/ storage/ innobase/ btr/btr0sea. c line 1374
131111 0:41:17 INNODB MONITOR OUTPUT
=======
Per second averages calculated from the last 15 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 36034 1_second, 36033 sleeps, 3601 10_second, 442 background, 442 flush
srv_master_thread log flush and writes: 37289
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 392362, signal count 406553
--Thread 22 has waited at row0purge.c line 682 for 246.00 seconds the semaphore:
Mutex at 27294d8a8 '&dict_sys->mutex', lock var 1
waiters flag 1
--Thread 27 has waited at btr0sea.c line 1778 for 0.0000 seconds the semaphore:
X-lock (wait_ex) on RW-latch at e52d77d8 'btr_search_
a writer (thread id 27) has reserved it in mode wait exclusive
number of readers 1, waiters flag 0, lock_word: ffffffffffffffff
Last time read locked in file btr0sea.c line 1420
Last time write locked in file /var/tmp/
Mutex spin waits 6117445, rounds 21430759, OS waits 326581
RW-shared spins 54246, rounds 639853, OS waits 16738
RW-excl spins 253164, rounds 1848521, OS waits 44718
Spin rounds per wait: 3.50 mutex, 11.80 RW-shared, 7.30 RW-excl
Purge operation is waiting on the dict_sys->mutex to be able to do purge of an undo log record, while it appears that the dict_sys->mutex is being held as part of the data dictionary LRU eviction process:
- storage/ innobase/ dict/dict0dict. c remove_ from_cache( own(&(dict_ sys->mutex) ));
1940 void
1941 dict_index_
...
1953 ut_ad(mutex_