Crash due to a long semaphore wait possibly due to deadlocked thread
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Percona Server moved to https://jira.percona.com/projects/PS |
New
|
Undecided
|
Unassigned | ||
Bug Description
Percona Server version 5.6.30-76.3 though looking at 95c9a07c9037d27
Crash message:
InnoDB: Warning: a long semaphore wait:
--Thread 139871293855488 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139858268354304 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861602424576 has waited at btr0cur.cc line 591 for 570.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139859438765824 has waited at row0ins.cc line 1809 for 573.00 seconds the semaphore:
S-lock on RW-latch at 0x1362860 '&dict_
a writer (thread id 139874778728192) has reserved it in mode wait exclusive
number of readers 2, waiters flag 1, lock_word: fffffffffffffffe
Last time read locked in file row0ins.cc line 1809
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139863352051456 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139873745737472 has waited at btr0cur.cc line 591 for 577.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139870322583296 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139870204339968 has waited at btr0cur.cc line 591 for 491.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139865210529536 has waited at btr0cur.cc line 591 for 559.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861023811328 has waited at btr0cur.cc line 591 for 575.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861385324288 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139874130208512 has waited at btr0cur.cc line 591 for 524.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139870693488384 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139862725093120 has waited at btr0cur.cc line 582 for 579.00 seconds the semaphore:
X-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139871155242752 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139873551193856 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139864820860672 has waited at btr0cur.cc line 257 for 547.00 seconds the semaphore:
S-lock on RW-latch at 0x7f4bce111800 '&block->lock'
a writer (thread id 139874223687424) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file btr0cur.cc line 257
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139873452336896 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861270673152 has waited at btr0cur.cc line 582 for 579.00 seconds the semaphore:
X-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139871212357376 has waited at btr0cur.cc line 582 for 579.00 seconds the semaphore:
X-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139871405717248 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139873986377472 has waited at btr0cur.cc line 591 for 539.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371d05fd08 '&new_index->lock'
a writer (thread id 139861162825472) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139873650853632 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139859164329728 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139865936545536 has waited at btr0cur.cc line 591 for 505.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139859259213568 has waited at btr0cur.cc line 591 for 571.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139859559778048 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139866848495360 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139873453340416 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139858268755712 has waited at btr0cur.cc line 591 for 566.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139859951073024 has waited at btr0cur.cc line 591 for 557.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139871698495232 has waited at row0ins.cc line 1809 for 559.00 seconds the semaphore:
S-lock on RW-latch at 0x1362860 '&dict_
a writer (thread id 139874778728192) has reserved it in mode wait exclusive
number of readers 2, waiters flag 1, lock_word: fffffffffffffffe
Last time read locked in file row0ins.cc line 1809
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139872147707648 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139872416544512 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139864312948480 has waited at btr0cur.cc line 582 for 579.00 seconds the semaphore:
X-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139870898407168 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861658355456 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139866427680512 has waited at btr0cur.cc line 257 for 550.00 seconds the semaphore:
X-lock on RW-latch at 0x7f4bce111560 '&block->lock'
a writer (thread id 139862270523136) has reserved it in mode exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file btr0cur.cc line 257
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139874112628480 has waited at row0ins.cc line 1809 for 573.00 seconds the semaphore:
S-lock on RW-latch at 0x1362860 '&dict_
a writer (thread id 139874778728192) has reserved it in mode wait exclusive
number of readers 2, waiters flag 1, lock_word: fffffffffffffffe
Last time read locked in file row0ins.cc line 1809
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861764396800 has waited at btr0cur.cc line 591 for 572.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139866443032320 has waited at btr0cur.cc line 591 for 539.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
--Thread 139873990592256 has waited at row0undo.cc line 298 for 576.00 seconds the semaphore:
S-lock on RW-latch at 0x1362860 '&dict_
a writer (thread id 139874778728192) has reserved it in mode wait exclusive
number of readers 2, waiters flag 1, lock_word: fffffffffffffffe
Last time read locked in file row0ins.cc line 1809
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139870968706816 has waited at btr0cur.cc line 591 for 529.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139871203587840 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139975452006144 has waited at btr0cur.cc line 591 for 570.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139861837707008 has waited at btr0cur.cc line 591 for 578.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139874213893888 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
--Thread 139862362175232 has waited at btr0cur.cc line 591 for 579.00 seconds the semaphore:
S-lock on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/
InnoDB: Warning: a long semaphore wait:
InnoDB: ###### Diagnostic info printed to the standard error stream
InnoDB: Error: semaphore wait has lasted > 600 seconds
InnoDB: We intentionally crash the server, because it appears to be hung.
2016-10-21 20:30:44 7f36ccfff700 InnoDB: Assertion failure in thread 139873344288512 in file srv0srv.cc line 2148
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://
InnoDB: about forcing recovery.
20:30:44 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://
key_buffer_
read_buffer_
max_used_
max_threads=16386
thread_count=16385
connection_
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/usr/sbin/
/usr/sbin/
/lib/x86_
/lib/x86_
/lib/x86_
/usr/sbin/
/lib/x86_
/lib/x86_
You may download the Percona Server operations manual by visiting
http://
in the manual which will help you identify the cause of the crash.
Locks:
41815 lock on RW-latch at 0x7f371e032508 '&new_index->lock'
4875 lock on RW-latch at 0x7f371d05fd08 '&new_index->lock'
3592 lock on RW-latch at 0x1362860 '&dict_
973 lock on RW-latch at 0x7f460b71a8c0 '&block->lock'
754 lock on RW-latch at 0x7f4327bd81c0 '&block->lock'
676 lock on RW-latch at 0x7f4bcdd40d60 '&block->lock'
472 lock on RW-latch at 0x7f4bce111800 '&block->lock'
122 lock on RW-latch at 0x7f460fd80a60 '&block->lock'
116 lock on RW-latch at 0x7f460f6f3460 '&block->lock'
65 lock on RW-latch at 0x7f460f769660 '&block->lock'
53 lock on RW-latch at 0x7f4bce111aa0 '&block->lock'
37 lock on RW-latch at 0x7f460fd79440 '&block->lock'
35 lock on RW-latch at 0x7f460f7fac20 '&block->lock'
28 lock on RW-latch at 0x7f460fdb4540 '&block->lock'
24 lock on RW-latch at 0x7f460fde1720 '&block->lock'
23 lock on RW-latch at 0x7f460fcdc420 '&block->lock'
19 lock on RW-latch at 0x7f4bce111560 '&block->lock'
16 lock on RW-latch at 0x7f4bc5f4a5a0 '&block->lock'
16 lock on RW-latch at 0x7f460f766720 '&block->lock'
15 lock on RW-latch at 0x7f460fd7b900 '&block->lock'
14 lock on RW-latch at 0x7f4329ec45e0 '&block->lock'
12 lock on RW-latch at 0x7f460fd17520 '&block->lock'
11 lock on RW-latch at 0x7f48eedc4fc0 '&block->lock'
8 lock on RW-latch at 0x7f48ea578460 '&block->lock'
6 lock on RW-latch at 0x7f460f783d00 '&block->lock'
6 lock on RW-latch at 0x7f460f6f2f20 '&block->lock'
4 lock on RW-latch at 0x7f48eedc1de0 '&block->lock'
4 lock on RW-latch at 0x7f3a8edfb980 '&block->lock'
3 lock on RW-latch at 0x7f460fde5b60 '&block->lock'
2 lock on RW-latch at 0x7f460fd0cd20 '&block->lock'
Mutexes:
65328 Mutex at 0x7f373a800118 '&lock_
30151 Mutex at 0x7f373a800068 '&lock_sys->mutex'
6 Mutex at 0x7f4e93e92068 '&trx_sys->mutex'
3 Mutex at 0x134bc40 '&srv_innodb_
Waits at:
65004 lock0wait.cc line 223
45080 btr0cur.cc line 591
9267 lock0lock.cc line 6149
7433 lock0lock.cc line 4463
7205 lock0lock.cc line 6941
3902 lock0lock.cc line 6483
2381 row0ins.cc line 2373
2171 row0undo.cc line 298
2014 lock0wait.cc line 271
1656 btr0cur.cc line 582
1400 row0ins.cc line 1809
1199 btr0cur.cc line 257
311 lock0wait.cc line 77
274 lock0lock.cc line 6011
33 row0ins.cc line 2718
23 srv0srv.cc line 2653
23 dict0stats.cc line 2388
18 buf0flu.cc line 1078
16 btr0cur.cc line 367
13 lock0wait.cc line 510
11 lock0lock.cc line 6294
11 ha_innodb.cc line 4809
10 lock0wait.cc line 95
10 btr0pcur.cc line 431
8 lock0lock.cc line 2797
6 lock0lock.cc line 1248
5 lock0lock.cc line 6402
5 lock0lock.cc line 3516
4 row0sel.cc line 3226
3 read0read.ic line 117
2 srv0srv.cc line 1632
1 trx0trx.cc line 329
1 trx0trx.cc line 1052
1 srv0srv.cc line 1301
1 lock0lock.cc line 6154
Last time write locked:
46736 Last time write locked in file /mnt/workspace/
3617 Last time write locked in file /mnt/workspace/
2405 Last time write locked in file /mnt/workspace/
668 Last time write locked in file /mnt/workspace/
531 Last time write locked in file /mnt/workspace/
29 Last time write locked in file /mnt/workspace/
16 Last time write locked in file /mnt/workspace/
12 Last time write locked in file /mnt/workspace/
| Ovais Tariq (ovais-tariq) wrote | #1 |
| Laurynas Biveinis (laurynas-biveinis) wrote | #2 |
| Ovais Tariq (ovais-tariq) wrote | #3 |
| Ovais Tariq (ovais-tariq) wrote | #4 |
| Shahriyar Rzayev (rzayev-sehriyar) wrote | #5 |
Analysis:
- thread id 139868969027328 has reserved &new_index->lock in mode wait exclusive
- all other threads are waiting on mtr_s_lock(
Appears to be a deadlock.
The thread 139868969027328 already has a wait_ex lock on index->lock.
Now its waiting for X-lock &new_index->lock
So what I understand is happening is:
thread 139868969027328 acquires a X lock &(index->lock)
thread 139868969027328 tries to acquire X lock &new_index->lock but the lock is already taken by itself.
Example:
--Thread 139868969027328 has waited at btr0cur.cc line 582 for 579.00 seconds the semaphore:
X-lock (wait_ex) on RW-latch at 0x7f371e032508 '&new_index->lock'
a writer (thread id 139868969027328) has reserved it in mode wait exclusive
number of readers 42, waiters flag 1, lock_word: ffffffffffffffd6
Last time read locked in file btr0cur.cc line 591
Last time write locked in file /mnt/workspace/