Mysql crashed with signal 11

[Romana]

-------------------------------------------------------------------------------------------------
Version
-------------------------------------------------------------------------------------------------
Mysql Server version 5.6.25-73.1-log
CentOS release 6.8 (Final)

-------------------------------------------------------------------------------------------------
From Logs
-------------------------------------------------------------------------------------------------
18:52:23 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=486
max_threads=3002
thread_count=305
connection_count=305
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 3507970 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7f04b8aca000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f0a87098d00 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x2c)[0x8cea7c]
/usr/sbin/mysqld(handle_fatal_signal+0x461)[0x655fa1]
/lib64/libpthread.so.0(+0xf7e0)[0x7f0a8a0ef7e0]
/usr/lib64/mysql/plugin/ha_tokudb.so(+0x7db63)[0x7f0a424ccb63]
/usr/lib64/mysql/plugin/ha_tokudb.so(Z53toku_txn_manager_handle_snapshot_create_for_child_txnP7tokutxnP11txn_manager19_TXN_SNAPSHOT_TYPE+0x1b5)[0x7f0a424fcb05]
/usr/lib64/mysql/plugin/ha_tokudb.so(Z23toku_txn_begin_with_xidP7tokutxnPS0_P10tokulogger12txnid_pair_s19TXN_SNAPSHOT_TYPEP13_toku_db_txnbb+0x143)[0x7f0a42533f93]
/usr/lib64/mysql/plugin/ha_tokudb.so(Z14toku_txn_beginP13toku_db_envP13_toku_db_txnPS2_j+0x30b)[0x7f0a425343eb]
/usr/lib64/mysql/plugin/ha_tokudb.so(_ZN9ha_tokudb9write_rowEPh+0x5a5)[0x7f0a424b6465]
/usr/sbin/mysqld(_ZN7handler12ha_write_rowEPh+0x97)[0x5a08c7]
/usr/sbin/mysqld(Z12write_recordP3THDP5TABLEP9COPY_INFOS4+0x85)[0x6bf2e5]
/usr/sbin/mysqld(_ZN13select_insert9send_dataER4ListI4ItemE+0xb3)[0x6bfc03]
/usr/sbin/mysqld[0x6b98c0]
/usr/sbin/mysqld[0x6b4def]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x149)[0x6b5159]
/usr/sbin/mysqld(_ZN4JOIN4execEv+0x2b8)[0x6b4388]
/usr/sbin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x275)[0x6fdf05]
/usr/sbin/mysqld(_Z13handle_selectP3THDP13select_resultm+0x165)[0x6fe765]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x592c)[0x6db82c]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x5a8)[0x6dcf98]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x107b)[0x6de7ab]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x162)[0x6abc32]
/usr/sbin/mysqld(handle_one_connection+0x40)[0x6abd20]
/usr/sbin/mysqld(pfs_spawn_thread+0x143)[0xadf3e3]
/lib64/libpthread.so.0(+0x7aa1)[0x7f0a8a0e7aa1]
/lib64/libc.so.6(clone+0x6d)[0x7f0a88800aad]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f04b43c7010): is an invalid pointer
Connection ID (thread ID): 56287368
Status: NOT_KILLED

-------------------------------------------------------------------------------------------------
My.cnf
-------------------------------------------------------------------------------------------------
[mysqld]

server-id=1017490233

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

#default-storage-engine=TokuDB
default-storage-engine=InnoDB
tokudb_load_save_space = 1
tokudb_commit_sync = 1 #Non-default
tokudb_fsync_log_period = 1000
transaction-isolation = READ-COMMITTED

#log-bin = /var/lib/mysql/mysql-bin #Non-default
log-bin
relay-log=/var/lib/mysql/relay-bin
binlog_format = MIXED
expire_logs_days=4
#binlog_checksum=NONE #Necessary for 5.6->5.5 replication
#log_bin_use_v1_row_events=1 #Necessary for 5.6->5.5 replication

skip-slave-start
skip-name-resolve
group_concat_max_len=65535
replicate-ignore-db = mysql
log_slave_updates
slave_type_conversions = ALL_NON_LOSSY
slave-skip-errors=1505

max_connect_errors = 5000
connect_timeout = 60
character-set-server=utf8

sql_mode=NO_ENGINE_SUBSTITUTION,ALLOW_INVALID_DATES #Non-default
max_connections=3000
max_allowed_packet = 16M
binlog_cache_size = 2M
sort_buffer_size = 1M
tmpdir = /var/lib/mysql
slow_query_log
slow_query_log_file = /var/log/mysql_slow_queries.log
long_query_time = 10
open_files_limit = 32768
table_open_cache_instances = 16
table_open_cache = 8000

[mysql]
no-auto-rehash

[mysqld_safe]
thp-setting=never
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

-------------------------------------------------------------------------------------------------
Last Record From Slow Quires
-------------------------------------------------------------------------------------------------
# Time: 170124 13:52:23
# Query_time: 26.615445 Lock_time: 0.000585 Rows_sent: 0 Rows_examined: 3931115 Rows_affected: 45947
# Bytes_sent: 56
SET timestamp=1485283943;
insert IGNORE into temp_update_7664_11386 select supporter.supporter_KEY from supporter left join email em41518 use index (email_blast_index) on supporter.supporter_KEY=em41518.supporter_KEY where supporter.organization_KEY=30047 and em41518.email_blast_KEY in (8509,8511,8510,8508,8507,8501,8500,8506,8505,8504,8499,8498,8503,8502,8495,8497,8496,8492,8494,8493,8489,8491,8490,8486,8488,8487,8485,8484,8480,8483,8482,8481,8476,8477,8479,8478,8475,8474,8470,8469,8473,8468,8472,8471,8465,8467,8466,8456,8460,8459,8454,8453,8452,8451,8448,8450,8449,8447,8446,8445,8440,8439,8438,8437,8436,8444,8443,8435,8442,8441,8434,8433,8432,8429,8431,8430,8425,8427,8426,8419,8421,8420,8418,8417,8414,8416,8415,8412,8413,8408,8407,8406,8410,8409,8402,8405,8404,8401,8400,8399,8398,8397,8396,8390,8393,8387,8395,8394,8386,8392,8391,8385,8389,8388,8378,8382,8384,8383,8379,8381,8380,8374,8376,8375,8371,8373,8372,8370,8369,8366,8368,8367,8362,8365,8364,8363,8359,8361,8360,8358,8355,8357,8356,8351,8353,8352,8348,8350,8349,8347,8344,8346,8345,8343,8340,8339,8342,8341,8335,8337,8336,8334,8331,8330,8333,8332,8326,8329,8328,8327,8316,8324,8323,8320,8322,8321,8317,8319,8318,8313,8315,8314,8310,8312,8311,8309) and em41518.status in ('Sent and Opened') UNION select supporter.supporter_KEY from supporter where supporter.organization_KEY=30047 and supporter.Date_Created > '2016-08-01 00:00';

NOTE: Manual rerun or this query was successful:
Query OK, 45950 rows affected (34.95 sec)
Records: 45950 Duplicates: 0 Warnings: 0

-------------------------------------------------------------------------------------------------
Others
-------------------------------------------------------------------------------------------------
We dont have general logs and mysql crash dump on

[George Ormond Lorch III (gl-az)]

Not enough information to reproduce or diagnose.

[George Ormond Lorch III (gl-az)]

If it occurs again and there is more information, please create a new issue in the new Percona TokuDB JIRA project located here : https://jira.percona.com/projects/TDB/issues/TDB-28

[Launchpad Janitor (janitor)]

[Expired for Percona Server because there has been no activity for 60 days.]

[Launchpad Janitor (janitor)]

[Expired for Percona Server 5.6 because there has been no activity for 60 days.]

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-3628