Kernel memory leak introduced with 14.04 kernel when used with squid3

[pg]

I have a Ubuntu 12.04.5 LTS running in a Hyper-V VM for several years.
Since I updated the system a few days ago to Linux 3.13.0-68-generic x86_64, the machine hangs
with out of memory after about 2 days of uptime. (occurred several times)

With the kernel from July (3.13.0-4?-generic x86_64 or 3.13.0-5? - I don't remember) there was no problem
running the machine several month without interruption.

I'm currently monitoring /proc/meminfo and /proc/slabinfo and can sea a steady increase of
Slab in meminfo.

I also have a second VM without showing the problem. The VM with the leak is used as internet
router and firewall. Two things apply to the VM the the other does not have:
- vsftpd is running, some files are downloaded from the machine every few minutes to hours.
- there is a "tcpdump -i eth0 -s 65535 -C 100 -W 1000000000 -w /somefile" running all the time.

The attached zip file contains the hourly logs of /proc/meminfo and /proc/slabinfo;
*_151115_171444 are the first files after the last reboot.

PS: Initially reported as Question: https://answers.launchpad.net/ubuntu/+question/274231

[pg (pg-ubuntu)]

This are the logs of the next 22 hours.
Slab is increasing about 20 MB per day; depending on the amount of network traffic.

[pg (pg-ubuntu)]

Correcting package

[pg (pg-ubuntu)]

I have now switch to a older kernel (3.13.0-57) and there is not leak there.
Therefore the leak has been introduced between 3.13.0-57 and 3.13.0-68.

[adam (fengsheng-10)]

Same issue for me.

We met squid3 (3.3.8 and 3.5.11) memory usage issue with ubuntu 14.04 in production environment.

and we prepared environment for testing, 3.3.8，3.5.11，2.7 on ubuntu 12.04 and 14.04.

3.3.8，3.5.11 on ubuntu 12.04 is ok
2.7 on ubuntu 14.04 is ok
3.3.8，3.5.11 on ubuntu 14.04 is abnormal.

and I downgraded kernel version to 3.13.0-57 on ubuntu 14.04, squid3 memory usage is normal now.

I don't know how was it introduced into kernel on ubuntu 14.04, but it is really works for me.

[adam (fengsheng-10)]

I think not only the kernel, also include libs squid depends on.
Try to rebuild squid.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robie Basak (racb)]

In duplicate bug 1521086 the 14.04 kernel is implicated, so adding a linux task.

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1516738

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[pg (pg-ubuntu)]

Unfortunately "apport-collect 1516738" does not work because I don't have a web browser accessible.

I've run "apport-bug linux-generic-lts-trusty" instead an attached the created file.
Please note, that the machine is currently running with 3.13.0-57 because I had do downgrade; 3.13.0-68 is installed however.

Let me know if the apport-bug has to be run with 3.13.0-68; it would take so time to find a quiet slot where I can reboot the machine.

[penalvch (penalvch)]

pg, thank you for reporting this and helping make Ubuntu better.

Could you please test the latest upstream kernel available from the very top line at the top of the page from http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D (the release names are irrelevant for testing, and please do not test the daily folder)? Install instructions are available at https://wiki.ubuntu.com/Kernel/MainlineBuilds . This will allow additional upstream developers to examine the issue.

If the latest kernel did not allow you to test to the issue (ex. you couldn't boot into the OS) please make a comment in your report about this, and continue to test the next most recent kernel version until you can test to the issue. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this issue is fixed in the mainline kernel, please add the following tags by clicking on the yellow circle with a black pencil icon, next to the word Tags, located at the bottom of the report description:
kernel-fixed-upstream
kernel-fixed-upstream-X.Y-rcZ

Where X, and Y are the first two numbers of the kernel version, and Z is the release candidate number if it exists.

If the mainline kernel does not fix the issue, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-X.Y-rcZ

Please note, an error to install the kernel does not fit the criteria of kernel-bug-exists-upstream.

Once testing of the latest upstream kernel is complete, please mark this report's Status as Confirmed. Please let us know your results.

Thank you for your understanding.

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]

[Enih (nanericwang)]

Please read this:
http://www.squid-cache.org/Doc/config/memory_pools/

To improve memory usage/performance of squid, please load jemalloc and turn off memory_pools.