Cannot change the maximum number of filedescriptors

[Emanuele Rocca]

The max_filedescriptors configuration directive is being ignored on Trusty.

To reproduce the problem, set max_filedescriptors in squid.conf and run the following command:

squid3 -k parse 2>&1|grep max_filedescriptors

The following output should result.

2014/11/06 15:21:15| WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.

Indeed, regardless of the max_filedescriptors value specified in squid.conf, squid will run with a maximum of 65536 file descriptors available. This can be verified with:

grep 'file descriptor' /var/log/squid3/cache.log
2014/11/06 15:30:45| With 65536 file descriptors available

The reason for this is that squid has been built with --with-filedescriptors=65536

Rebuilding the package without --with-filedescriptors, squid honors the maximum number of open file descriptors set with ulimit.

Fedora also had a similar problem:
https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=976815

Thanks,
  Emanuele

apt-cache policy squid3 | grep Installed
  Installed: 3.3.8-1ubuntu6.1

lsb_release -rd
Description: Ubuntu 14.04 LTS
Release: 14.04

[Zing (zing-redhat-bugs)]

Version-Release number of selected component (if applicable):
squid-3.2.9-1.fc18.x86_64

It's not possible to change the number of file descriptors available to squid at runtime with either the "max_filedescriptors" option or by specifying the open files limit in systemd squid unit file.

Attempting to use max_filedescriptors option gets an error:
WARNING: max_filedescriptors disabled. Operating System setrlimit(RLIMIT_NOFILE) is missing.

Rebuilding squid without the "--with-filedescriptors=16384" option in the spec file causes squid to inherit the build environments open files limit as its default (i.e. 1024 as a user), but the squid unit file option "LimitNOFILE=16384" is now obeyed and set if systemd is used to start squid. The max_filedescriptor option also starts working at this point.

Can we get squid built without the "--with-filedescriptors=16384" build option. Thank you.

[Michal (michal-redhat-bugs)]

It would make sense to compile squid without the file descriptors limit and treat systemd LimitNOFILE as a boundary for config value, but there is an issue when squid.conf value is higher than LimitNOFILE. In this case squid logs and probably also uses max_filedescriptor value instead of current fd limit.

I will initiate discussion with upstream on the theme of systemd and squid.

[Zing (zing-redhat-bugs)]

Yes. From my testing (with a squid built without the --with-filedescriptors option), if max_filedescriptors is not set, squid will just inherit the current rlimit setting of nofile (I made an error in my comment 0, it does not come from the build environment). If max_filedescriptors is set, then that is what squid will use, regardless of the nofile rlimit.

Is that necessarily an issue? That seems "ok" to me and the effect I'd want (being able to "override" the rlimit in the squid config).

The only kind of maybe unexpected thing here is if one doesn't use systemd to start squid, the default squid configuration is probably going to inherit a filedescriptor limit much smaller (1024) than the systemd LimitNOFILE specified 16384.

[Michal (michal-redhat-bugs)]

You can find the newest Squid build without "--with-filedescriptors" option in rawhide.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Amos Jeffries (yadi)]

This was fixed upstream in 3.3.11. So the Squid v3.5 package in Xenial and later is fixed.

[Amos Jeffries (yadi)]

For the record upstream patch is at http://bugs.squid-cache.org/show_bug.cgi?id=3970 but requires the build system to be re-bootstrapped so it cannot be easily applied to downstream packages.

[Amos Jeffries (yadi)]

Oops, that was the bug. I mean the patch is http://www.squid-cache.org/Versions/v3/3.3/changesets/squid-3.3-12662.patch