rsyslog starts dropping connections in large environments

[James Page]

256 server MAAS environment on hardware.

rsyslog on bootstrap node starts to drop connections due to concurrency:

May 11 17:08:03 sm15k-1-03 rsyslogd-2079: too many tcp sessions - dropping incoming request [try http://www.rsyslog.com/e/2079 ]

This is probably a ulimit hit in rsyslog configuration.

[Menno Finlay-Smits (menno.smits)]

This sounds like there needs to a tweak to the rsyslogd process limits to allow more open file handles and/or a change to the maximum number of connections in rsyslogd's config.

[Menno Finlay-Smits (menno.smits)]

It's worth noting that this is unlikely to be a problem with Juju 2.0 as we no longer have all the agents in each environment logging to rsyslogd. Juju agent logs are now sent to the controller machines over the Juju API and are stored in MongoDB.

This is really just a Juju 1.25 issue. Retargetting...