OOM on server after installing 7.0.33-0ubuntu0.16.04.11 update

Bug #1863850 reported by Mario Jansen on 2020-02-19
264
This bug affects 2 people
Affects Status Importance Assigned to Milestone
php7.0 (Ubuntu)
Critical
Leonidas S. Barbosa

Bug Description

After installing te update 7.0.33-0ubuntu0.16.04.11 on our servers, php is taking more memory (until the server is completly out of memory).
The update was installed : 2020-02-18 02:15:45
when the server had an memory footprint of 2.1GB free of 3.8GB total.
2020-02-18 07:50 the server had still 98.6M free of 3.8 GB in total

Description: Ubuntu 16.04.6 LTS
Release: 16.04

Running 193 fpm pools with one master.
#ls -la /etc/php/7.0/fpm/pool.d/| wc -l
193

/etc/php/7.0/fpm/pool.d/user1.conf
fpm pool configuration:
[user1]
user = user1
group = user1
listen = /var/run/user1-fpm.sock
listen.owner = www-data
listen.group = www-data
chdir = /
pm = ondemand
pm.max_children = 5
pm.max_requests = 50
php_admin_value[memory_limit] = 256M
php_admin_value[error_log] = /var/log/homediruser1/phperror.log

Thist issue occured on multiple machines (same configuration). After rolling back this update the issue was gone.

Another machine got out of memory way sooner. This one has only 91 fpm pools / websites. This machine is in general takes a little more resources.

On our 18.04 LTS servers with the same setup this issue (with update php7.2-fpm:amd64 7.2.24-0ubuntu0.18.04.3) does not seem to occur.

Leonidas S. Barbosa (leosilvab) wrote :

Hi Mario,

Thanks for report this issue. Could you please show steps in how to reproduce that issue?

Tks!

Leonidas S. Barbosa (leosilvab) wrote :

I did push a regression version without the patch that touches fpm (possibly the offend one). Could you please test with this version, soon it's finished in: https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages?field.name_filter=php7.0&field.status_filter=published&field.series_filter=

version is: php7.0 - 7.0.33-0ubuntu0.16.04.12

Leonidas,

How can I safely test this change? The problem revealed itsself only on
our production servers (probabely because this machines are under more
workload). I will check when we can test this patch on a server during
non-peak hours.

Can I test and rollback this via adding the security proposed ppa?

add-apt-repository ppa:ubuntu-security-proposed/ppa && apt-get update

Regards,

Mario

On 2/19/20 3:29 PM, Leonidas S. Barbosa wrote:
> I did push a regression version without the patch that touches fpm
> (possibly the offend one). Could you please test with this version, soon
> it's finished in: https://launchpad.net/~ubuntu-security-
> proposed/+archive/ubuntu/ppa/+packages?field.name_filter=php7.0&field.status_filter=published&field.series_filter=
>
> version is: php7.0 - 7.0.33-0ubuntu0.16.04.12
>

Leonidas S. Barbosa (leosilvab) wrote :

It's better you just install it by hand since once you add this repo you will get more than just php updates/upgrade.

Changed in php7.0 (Ubuntu):
status: New → Confirmed
assignee: nobody → Leonidas S. Barbosa (leosilvab)
importance: Undecided → Critical
Leonidas S. Barbosa (leosilvab) wrote :

I just pushed a new version removing the patch you can grab it by normal apt-get update; apt-get upgrade.

Mario Jansen (mariojansen) wrote :

I don't see an memory usage increase after installing the update. The
issue seems to be solved.

Thanks for the fast response.

On 2/19/20 7:58 PM, Leonidas S. Barbosa wrote:
> I just pushed a new version removing the patch you can grab it by normal
> apt-get update; apt-get upgrade.
>

Changed in php7.0 (Ubuntu):
status: Confirmed → Fix Released
information type: Private Security → Public Security
Simon Déziel (sdeziel) on 2020-02-21
tags: added: snap
tags: removed: snap
tags: added: regression-update
tags: added: xenial
To post a comment you must log in.
This report contains Public Security information  Edit
Everyone can see this security related information.

Duplicates of this bug

Other bug subscribers