HAProxy maxconns needs full restart (not reload)

Bug #1867063 reported by Haw Loeung
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Content Cache Charm
Fix Released
Medium
Haw Loeung

Bug Description

Hi,

As seen today, when increasing maxconns, a full HAProxy restart is required in order for it to increase the ulimit max. fds / open files. A reload respawns the child process but the master still has that limit and is inherited.

We should either restart or flag that it needs restarting and for admins to run a juju action to restart it. This should be done only if the ulimit for processes are lower than the global maxconns.

Or maybe even override the systemd service setting it to unlimited for HAProxy (LimitNOFILE=infinity or similar).

Related branches

Haw Loeung (hloeung)
description: updated
Haw Loeung (hloeung)
description: updated
Revision history for this message
Haw Loeung (hloeung) wrote :

Also figure out this one:

| Mar 11 21:47:38 cactuar haproxy[16470]: [WARNING] 070/214735 (16470) : [/usr/sbin/haproxy.main()] Cannot raise FD limit to 14592571, limit is 1048576.
| Mar 11 21:47:38 cactuar haproxy[16470]: [WARNING] 070/214735 (16470) : [/usr/sbin/haproxy.main()] FD limit (1048576) too low for maxconn=7296000/maxsock=14592571. Please raise 'ulimit-n' to 14592571 or more to avoid any trouble.

Revision history for this message
Haw Loeung (hloeung) wrote :

Okay, that 1048576 is from LXC/LXD.

Revision history for this message
Haw Loeung (hloeung) wrote :
Haw Loeung (hloeung)
Changed in content-cache-charm:
assignee: nobody → Haw Loeung (hloeung)
status: New → In Progress
Haw Loeung (hloeung)
Changed in content-cache-charm:
importance: Undecided → Medium
Revision history for this message
Haw Loeung (hloeung) wrote :

Linked MP:385341. Taking a different approach and using prlimit to increase the parent HAProxy process's max. open files. This saves downtime with having to restart HAProxy.

Revision history for this message
Haw Loeung (hloeung) wrote :

Looks like there's code in newer versions of HAProxy to already do this (setrlimit() in commits 304e17eb88, 304e17eb88, and probably others).

Haw Loeung (hloeung)
Changed in content-cache-charm:
status: In Progress → Fix Committed
Haw Loeung (hloeung)
Changed in content-cache-charm:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.