connection is refused while excuting "flush privileges"

Bug #966254 reported by Hui Liu
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MariaDB
New
Undecided
Unassigned
MySQL Server
Unknown
Unknown
Percona Server moved to https://jira.percona.com/projects/PS
Status tracked in 5.7
5.1
Won't Fix
Medium
Unassigned
5.5
Triaged
Medium
Unassigned
5.6
Triaged
Medium
Unassigned
5.7
Fix Released
Medium
Unassigned

Bug Description

In such scenario, we hit the infinite looping of 'flush privileges' events between master and
slave(it's also a master but read-only), which caused the app connections periodic error
of "host is not allowed to connect to this MySQL server".

As for the "flush privileges" passing between master and slave, back and forth, could refer to
https://bugs.launchpad.net/percona-server/+bug/940404

The root problems caused the connection issue is due to the logic defect which explained
as below:

For 'flush privileges', acl_reload()would be called, and then acl_load() is called. The
global variable allow_all_hosts will set to 0 under the lock, as well as the modification
for acl_check_hosts.

But when client connections to server, acl_check_hosts() will be called, which logic is
list below:

bool acl_check_host(const char *host, const char *ip)
1496 {
1497 if (allow_all_hosts)
1498 return 0;
1499 VOID(pthread_mutex_lock(&acl_cache->lock));
1500
1501 if ((host && hash_search(&acl_check_hosts,(uchar*) host,strlen(host))) ||
1502 (ip && hash_search(&acl_check_hosts,(uchar*) ip, strlen(ip))))
1503 {

in line 1497,we find that allow_all_hosts is used without any lock, which caused
inconsistent between allow_all_hosts and acl_check_hosts.

So there is a race risk, and in some special case which like we hit, that might be a
problem.

How to repeat:
NoN

Suggested fix:
check allow_all_hosts under the lock, just like this:

VOID(pthread_mutex_lock(&acl_cache->lock));
if (allow_all_hosts){
   VOID(pthread_mutex_unlock(&acl_cache->lock));
   return 0;
}

Revision history for this message
yinfeng (yinfeng-zwx) wrote :

this bug is very easy to repeat

1.create user xx1

root@(none) 05:10:33>grant all on *.* to xx1@'127.0.0.1' identified by '123';
Query OK, 0 rows affected (0.00 sec)

2. run sh:

$cat flush_pri.sh
#!/bin/sh

for i in {1..100000}; do
mysql -uroot -e 'flush privileges' &
mysql -uxx1 -p123 -h127.0.0.1 -P3320 -e "select user()"
done

3.
$sh flush_pri.sh | grep 'host'
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host '127.0.0.1' is not allowed to connect to this MySQL server
……
……
……

Revision history for this message
yinfeng (yinfeng-zwx) wrote :

a simple patch for this bug.
based on Percona 5.5.18

Stewart Smith (stewart)
Changed in percona-server:
importance: Undecided → Medium
Revision history for this message
yinfeng (yinfeng-zwx) wrote :

http://bugs.mysql.com/bug.php?id=63178

this bug was fixed in MySQL 5.7.0

Noted in 5.7.0 changelog.

The server refused client connections while executing FLUSH
PRIVILEGES

Changed in maria:
milestone: none → 5.1
Changed in percona-server:
status: New → Confirmed
Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

 ./flush.sh | grep 'not allowed to '
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server
ERROR 1130 (HY000): Host 'localhost' is not allowed to connect to this MySQL server

tags: added: contribution
Revision history for this message
Sergei Golubchik (sergii) wrote :
tags: added: upstream
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1243

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.