Multi-Master Replication Manager for MySQL mmm_agentd Remote Command Injection Vulnerabilities

[Talos Security Advisory]

Multiple exploitable remote command injection vulnerabilities exist
in the MySQL Master-Master Replication Manager (MMM) mmm_agentd
daemon 2.2.1. mmm_agentd commonly runs with root privileges and does not
require authentication by default.

[Talos Security Advisory (regiwils)]

This issue is beyond 90 days. Will it be assigned?

[Talos Security Advisory (regiwils)]

Issued copy of report to <email address hidden>

[Arjen Lentz (arjen-lentz)]

Thanks for reaching out - discussing strategy via email.

[Arjen Lentz (arjen-lentz)]

Hi Regina

> The Cisco Talos team found a security vulnerability impacting MySQL MMM
> customers. We issued a bug report (#1736962) via
> https://launchpad.net/mysql-mmm back in December and to date have not received
> a response. Can you assist with this issue? Attached is a copy of the issue
> in plaintext file.

Thanks for reaching out to me, much appreciated.

As indicated by the report, MySQL-MMM is definitely a deprecated tool, which may explain why the people on the security bugs list weren't responsive.
Still unfortunate though, since your team put a lot of effort and detail in the report.

In email, I've explicitly cc'd Pascal Hofmann (the Debian package maintainer) and Peter Zaitsev, CEO at Percona (Percona was one of the key original code maintainers).
(they would've been recipients from your original bug report)
[I'm also replying here, as the Talos mailserver appears to have an issue with our outbound mail system on IPv6 - I've filed a review request with Talos to rectify]

In terms of distro packages
 - Debian Jessie was the last version that had mysql-mmm packages
 - Ubuntu 16.04 LTS same. I've checked and the upcoming 18.04 does not contain mysql-mmm. Non-LTS already didn't contain mysql-mmm.
 - Fedora 27 still has mysql-mmm listed
 - CentOS 7 has mysql-mmm in EPEL.

Very few people would still be using mysql-mmm, I know of only one in our direct sphere.
We might still be able to see the issues fixed, particularly since you've provided a very specific list of the issues.
Please allow me to get back to you within the next few days with an opinion whether or not that's feasible.

If so, we can produce a patch (hopefully within a week or so), update things on Launchpad.net and submit it to Fedora/CentOS.
If we can't, the advisory should just be published and we can encourage Fedora to deprecate the package.

That ok?
thanks

[Arjen Lentz (arjen-lentz)]

David Beveridge is looking in to this, and is working on testing a patch.

[David Beveridge (dage)]

This patch adds some regex to validate that the supplied parameters are not dangerous, to ensure that the vulnerability can no longer be exploited. e.g. In the IP fields it looks only for digits and dots, so is only compatible with IPv4 Addresses. Testing revealed that mysql-mmm doesn't play well with IPv6. IPv6 really started to look like a feature addition so I decided to keep it out of this patch.

I also included a change to the SSL peer validation. Without the SSL_verify_mode change the following is displayed when using monitor commands when SSL is enabled;
# mmm_control show
*******************************************************************
 Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client
 is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER
 together with SSL_ca_file|SSL_ca_path for verification.
 If you really don't want to verify the certificate and keep the
 connection open to Man-In-The-Middle attacks please set
 SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.
*******************************************************************
  at /usr/share/perl5/vendor_perl/MMM/Common/Socket.pm line 86.

[David Beveridge (dage)]

TESTING

I have added the following to the configuration on /etc/mysql-mmm/mmm_common.conf on the mmm monitor machine

<role writer>
    hosts db1,db2
    ips 10.2.47.34, `/bin/killall mmm_agentd`
    mode exclusive
</role>

This causes the monitor to send a command that will expose the vulnerability.

On the unpatched database server running in debug mode I see this

root@db2 ~ # mmm_agentd
2018/04/25 05:34:11 DEBUG Created pid file '/run/mysql-mmm-agent.pid' with pid 11511
2018/04/25 05:34:11 DEBUG Listener: Waiting for connection...
2018/04/25 05:34:13 DEBUG Listener: Connect!
2018/04/25 05:34:13 DEBUG Daemon: Command = 'SET_STATUS|1|db2|ONLINE|reader(10.2.47.35),writer(`/bin/killall mmm_agentd`)|db2'
2018/04/25 05:34:13 DEBUG Received Command SET_STATUS|1|db2|ONLINE|reader(10.2.47.35),writer(`/bin/killall mmm_agentd`)|db2
2018/04/25 05:34:13 INFO We have some new roles added or old rules deleted!
2018/04/25 05:34:13 INFO Added: reader(10.2.47.35), writer(`/bin/killall mmm_agentd`)
2018/04/25 05:34:13 DEBUG Executing /usr/libexec/mysql-mmm//agent/configure_ip mmm_agent ens192 10.2.47.35
2018/04/25 05:34:13 DEBUG Executing /usr/libexec/mysql-mmm//agent/sync_with_master mmm_agent
2018/04/25 05:34:13 DEBUG Executing /usr/libexec/mysql-mmm//agent/mysql_allow_write mmm_agent
2018/04/25 05:34:13 DEBUG Executing /usr/libexec/mysql-mmm//agent/configure_ip mmm_agent ens192 `/bin/killall mmm_agentd`
2018/04/25 05:34:13 INFO Signal received: exiting...
2018/04/25 05:34:13 INFO Signal received: exiting...
2018/04/25 05:34:13 FATAL Couldn't configure IP '`/bin/killall mmm_agentd`' on interface 'ens192': Usage: /usr/libexec/mysql-mmm//agent/configure_ip <config_file> <interface> <ip>
2018/04/25 05:34:13 DEBUG Fetching uptime from /proc/uptime
2018/04/25 05:34:13 DEBUG Uptime is 232654.20
2018/04/25 05:34:13 DEBUG Daemon: Answer = 'OK: Status applied successfully!'
2018/04/25 05:34:13 INFO END
2018/04/25 05:34:13 INFO Child exited normally (with exitcode 0), shutting down

The agent shuts down after it executes the backtick command.

Here we see the patched agent running with the same setup.

2018/04/25 05:31:46 DEBUG Listener: Waiting for connection...
2018/04/25 05:31:49 DEBUG Listener: Connect!
2018/04/25 05:31:49 DEBUG Daemon: Command = 'SET_STATUS|1|db2|ONLINE|reader(10.2.47.35),writer(`/bin/killall mmm_agentd`)|db2'
2018/04/25 05:31:49 DEBUG Received Command SET_STATUS|1|db2|ONLINE|reader(10.2.47.35),writer(`/bin/killall mmm_agentd`)|db2
2018/04/25 05:31:49 DEBUG Fetching uptime from /proc/uptime
2018/04/25 05:31:49 DEBUG Uptime is 232509.65
2018/04/25 05:31:49 DEBUG Daemon: Answer = 'OK: Status applied successfully!'
2018/04/25 05:31:49 DEBUG Listener: Disconnect!
2018/04/25 05:31:49 DEBUG Executing /usr/libexec/mysql-mmm//agent/configure_ip mmm_agent ens192 10.2.47.35
2018/04/25 05:31:49 DEBUG Listener: Waiting for connection...
2018/04/25 05:31:52 DEBUG Listener: Connect!
2018/04/25 05:31:52 DEBUG Daemon: Command = 'SET_STATUS|1|db2|ONLINE|reader(10.2.47.35),writer(`/bin/killall mmm_agentd`)|db2'
2018/04/25 05:31:52 DEBUG Received Command SET_STATUS|1|db2|ONLINE|reader(10.2.47.35),writer(`/bin/killall mmm_agentd`)|db2
2018/04/2...

[Talos Security Advisory (regiwils)]

Is there a confirmed fix/planned public release date?