rsyslog hangs if setuid during logging process causes further logging

[Stephen Parry]

I have a server which uses slapd and libnss_ldap based authentication through nsswitch.conf as per the manuals. After upgrading to 14.04 we lost our logging altogether; syslog was un-installed. After first installing syslog-ng I switched to using rsyslogd. The server hung consistently on boot. After several days of debugging, I have finally identified how and where the problem manifests.
When any process starts logging, rsyslog setgids and setuids as specified in rsyslog.conf. The setgid generally works OK (it is already running that group id) the setuid calls nss which then attempts to contact LDAP. LDAP is not yet running and cannot at that point in the startup. nss then tries to log a warning re-entering back into rsyslog. This locks up rsyslog, the calling process and any subsequent calling processes. boom.

In the attached apport I have simulated what happens using the logger command with rsyslog running but slapd stopped.

[Stephen Parry (sgparry)]

I found a workaround that shuts libnss up; the following settings in /etc/ldap.conf were:

    nss_initgroups_minimum_uid 0
    nss_initgroups_ignoreusers root

I changed:

    nss_initgroups_minimum_uid 30

then used usermodify to shuffle the syslog user uid (it was 120ish):

    usermodify -u14 syslog
    groupmodify -g14 syslog

I then re-populated nss_initgroups_ignore_users using:

    service libnss-ldap restart

giving:

    nss_initgroups_ignoreusers
bin,daemon,games,lp,mail,man,news,proxy,root,sync,sys,syslog,uucp

This shut libnss_ldap up and stopped the log from hanging.