modifying permissions while the daemon is runnign gives strange behavior

[Walter Heck]

One of our slaves was acting up. Turns out that sometime in the past, someone accidentally made the mmm config files world-readable. MMM started failing silently, and as a result this slave never switched masters. In mmm_control however, it showed up perfectly healthy!
A slave should not show up healthy in mmm_control show when it has the wrong master!

[root@server03 ~]# tail /var/log/mysql-mmm/mmm_agentd.log
2011/01/19 23:05:23 INFO Changing active master to 'server02'
2011/01/19 23:05:23 FATAL Failed to change master to 'server02': undef
2011/01/19 23:05:26 INFO Changing active master to 'server02'
2011/01/19 23:05:26 FATAL Failed to change master to 'server02': undef
2011/01/19 23:05:29 INFO Changing active master to 'server02'
2011/01/19 23:05:29 FATAL Failed to change master to 'server02': undef
2011/01/19 23:05:32 INFO Changing active master to 'server02'
2011/01/19 23:05:32 FATAL Failed to change master to 'server02': undef
2011/01/19 23:05:35 INFO Changing active master to 'server02'
2011/01/19 23:05:35 FATAL Failed to change master to 'server02': undef
[root@server03 ~]# service mysql-mmm-agent stop
Stopping MMM Agent Daemon: [ OK ]
[root@server03 ~]# service mysql-mmm-agent start
Starting MMM Agent Daemon: Configuration file /etc/mysql-mmm/mmm_agent.conf is world readable! at /usr/lib/perl5/vendor_perl/5.8.8/MMM/Common/Config.pm line 132
                                                           [FAILED]
[root@server03 ~]# chmod go-rwx /etc/mysql-mmm/mmm_agent.conf
[root@server03 ~]# service mysql-mmm-agent start
Starting MMM Agent Daemon: Configuration file /etc/mysql-mmm/mmm_common.conf is world readable! at /usr/lib/perl5/vendor_perl/5.8.8/MMM/Common/Config.pm line 132
                                                           [FAILED]
[root@server03 ~]# chmod go-rwx /etc/mysql-mmm/mmm_common.conf
[root@server03 ~]# service mysql-mmm-agent start
Starting MMM Agent Daemon: [ OK ]
[root@server03 ~]# tail /var/log/mysql-mmm/mmm_agentd.log
2011/01/19 23:06:14 INFO Changing active master to 'server02'
2011/01/19 23:06:14 FATAL Failed to change master to 'server02': undef
2011/01/19 23:06:15 INFO Signal received: exiting...
2011/01/19 23:06:15 INFO END
2011/01/19 23:06:15 INFO Child exited normally (with exitcode 0), shutting down
2011/01/19 23:06:21 FATAL Configuration file /etc/mysql-mmm/mmm_agent.conf is world readable!
2011/01/19 23:06:38 FATAL Configuration file /etc/mysql-mmm/mmm_common.conf is world readable!
2011/01/19 23:06:56 INFO Changing active master to 'server02'
2011/01/19 23:06:56 INFO We have some new roles added or old rules deleted!
2011/01/19 23:06:56 INFO Added: reader(192.168.100.121)
[root@server03 ~]# grep chmod /home/openquery/.bash_history
sudo chmod -R o+r /etc/mysql-mmm
[root@server03 ~]#