Memory leak when a blind CONNECT tunnel job is closed
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Squid |
Unknown
|
Unknown
|
|||
squid (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Jammy |
Fix Released
|
Undecided
|
Sergio Durigan Junior |
Bug Description
[ Impact ]
Squid users can experience memory leaks when a "lonely" to-server connection closes, because blind CONNECT tunnel jobs are not being destroyed in this scenario. In other words, if a connection to the server gets closed, squid fails to close its associated tunnel. This memory leak would build up after some time leading to squid being OOM killed.
This regression was introduced by https:/
[ Test Plan ]
Unfortunately, this is one of those bugs that require a non-trivial amount of time *and* setup to trigger. The reporter has been very kind and helpful, was able to consistently reproduce the bug after a few days of leaving squid running, and kindly tested the proposed patch to make sure it works. After 11 days without seeing the bug manifest again, I decided that it's justifiable to proceed with the SRU and rely on the reporter's help to officially verify that the problem has indeed been addressed.
[ Where problems could occur ]
The patch is extremely simple: it just calls "retryOrBail" as the last step of the function that is notified when the server closes the tunnel. The "retryOrBail" function acts almost like a destructor, making sure to either retry/reforward the connection if applicable, or to bail and delete the resources otherwise.
Although this code has been running in production for several months without regressions (as can be seen in the upstream bug's comments), there is still the (small) chance that a problematic situation arises due to this extra function call, especially considering that it deals with volatile situations like clients and servers closing a connection, sometimes abruptly. In the unlikely event that we encounter such problems in the future, we can revert the patch and get in touch with upstream, who has always been very responsive and competent in fixing complex bugs.
[ Original Description ]
There is a known memory leak in squid 5.2 - as noted here: https:/
The only readily available version of squid in the latest LTS release of Ubuntu (22.04) is 5.2-1ubuntu4.1, which means that doing a release upgrade leads to Ubuntu web proxies having an unstable instance of squid, as it gets OOM killed.
See, for example, another user with an issue: https:/
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu82.1
Architecture: amd64
CasperMD5CheckR
DistroRelease: Ubuntu 22.04
InstallationDate: Installed on 2021-11-22 (294 days ago)
InstallationMedia: Ubuntu-Server 20.04.3 LTS "Focal Fossa" - Release amd64 (20210824)
Package: squid 5.2-1ubuntu4.1
PackageArchitec
ProcEnviron:
TERM=xterm-
PATH=(custom, no user)
XDG_RUNTIME_
LANG=en_GB.UTF-8
SHELL=/bin/bash
ProcVersionSign
Tags: jammy uec-images
Uname: Linux 5.15.0-47-generic x86_64
UpgradeStatus: Upgraded to jammy on 2022-08-26 (17 days ago)
UserGroups: N/A
_MarkForUpload: True
mtime.conffile.
Related branches
- git-ubuntu bot: Approve
- Bryce Harrington (community): Approve
- Canonical Server Reporter: Pending requested
-
Diff: 76 lines (+54/-0)3 files modifieddebian/changelog (+8/-0)
debian/patches/close-tunnel-if-to-server-conn-closes-after-client.patch (+45/-0)
debian/patches/series (+1/-0)
summary: |
- squid memory leak + squid 5.2 memory leak - affects Ubuntu LTS 22.04 |
description: | updated |
Changed in squid (Ubuntu Jammy): | |
status: | Confirmed → In Progress |
Thank you for taking the time to report this bug and helping to make Ubuntu better.
According to the upstream bug, this is the upstream commit to master: https:/ /github. com/squid- cache/squid/ commit/ 752fa2083698a88 533ef23d88490f4 3f153f7a4c
It looks like this was fixed in 5.5-1, so marking Fix Released for Kinetic, and opening a task for Jammy.
https:/ /github. com/squid- cache/squid/ commit/ 54ad10efe146c86 3bdadee0d116129 9ea89f5419 looks like the cherry-pick to the upstream v5 branch.
@phyphor it would be helpful if you could provide steps to reproduce the problem on 22.04 please, to help validate the fix.