Ubuntu
php7.0 package

OOM on server after installing 7.0.33-0ubuntu0.16.04.11 update

Bug #1863850 reported by Mario Jansen
264
This bug affects 2 people
Affects Status Importance Assigned to Milestone
php7.0 (Ubuntu)
Fix Released
Critical
Leonidas S. Barbosa

Bug Description

After installing te update 7.0.33-0ubuntu0.16.04.11 on our servers, php is taking more memory (until the server is completly out of memory).
The update was installed : 2020-02-18 02:15:45
when the server had an memory footprint of 2.1GB free of 3.8GB total.
2020-02-18 07:50 the server had still 98.6M free of 3.8 GB in total

Description: Ubuntu 16.04.6 LTS
Release: 16.04

Running 193 fpm pools with one master.
#ls -la /etc/php/7.0/fpm/pool.d/| wc -l
193

/etc/php/7.0/fpm/pool.d/user1.conf
fpm pool configuration:
[user1]
user = user1
group = user1
listen = /var/run/user1-fpm.sock
listen.owner = www-data
listen.group = www-data
chdir = /
pm = ondemand
pm.max_children = 5
pm.max_requests = 50
php_admin_value[memory_limit] = 256M
php_admin_value[error_log] = /var/log/homediruser1/phperror.log

Thist issue occured on multiple machines (same configuration). After rolling back this update the issue was gone.

Another machine got out of memory way sooner. This one has only 91 fpm pools / websites. This machine is in general takes a little more resources.

On our 18.04 LTS servers with the same setup this issue (with update php7.2-fpm:amd64 7.2.24-0ubuntu0.18.04.3) does not seem to occur.

Revision history for this message
Leonidas S. Barbosa (leosilvab) wrote :

Hi Mario,

Thanks for report this issue. Could you please show steps in how to reproduce that issue?

Tks!

Revision history for this message
Leonidas S. Barbosa (leosilvab) wrote :

I did push a regression version without the patch that touches fpm (possibly the offend one). Could you please test with this version, soon it's finished in: https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages?field.name_filter=php7.0&field.status_filter=published&field.series_filter=

version is: php7.0 - 7.0.33-0ubuntu0.16.04.12

Revision history for this message
Mario Jansen (mariojansen) wrote : Re: [Bug 1863850] Re: OOM on server after installing 7.0.33-0ubuntu0.16.04.11 update

Leonidas,

How can I safely test this change? The problem revealed itsself only on
our production servers (probabely because this machines are under more
workload). I will check when we can test this patch on a server during
non-peak hours.

Can I test and rollback this via adding the security proposed ppa?

add-apt-repository ppa:ubuntu-security-proposed/ppa && apt-get update

Regards,

Mario

On 2/19/20 3:29 PM, Leonidas S. Barbosa wrote:
> I did push a regression version without the patch that touches fpm
> (possibly the offend one). Could you please test with this version, soon
> it's finished in: https://launchpad.net/~ubuntu-security-
> proposed/+archive/ubuntu/ppa/+packages?field.name_filter=php7.0&field.status_filter=published&field.series_filter=
>
> version is: php7.0 - 7.0.33-0ubuntu0.16.04.12
>

Revision history for this message
Leonidas S. Barbosa (leosilvab) wrote :

It's better you just install it by hand since once you add this repo you will get more than just php updates/upgrade.

Changed in php7.0 (Ubuntu):
status: New → Confirmed
assignee: nobody → Leonidas S. Barbosa (leosilvab)
importance: Undecided → Critical
Revision history for this message
Leonidas S. Barbosa (leosilvab) wrote :

I just pushed a new version removing the patch you can grab it by normal apt-get update; apt-get upgrade.

Revision history for this message
Mario Jansen (mariojansen) wrote :

I don't see an memory usage increase after installing the update. The
issue seems to be solved.

Thanks for the fast response.

On 2/19/20 7:58 PM, Leonidas S. Barbosa wrote:
> I just pushed a new version removing the patch you can grab it by normal
> apt-get update; apt-get upgrade.
>

Leonidas S. Barbosa (leosilvab)
Changed in php7.0 (Ubuntu):
status: Confirmed → Fix Released
information type: Private Security → Public Security
Simon Déziel (sdeziel)
tags: added: snap
tags: removed: snap
Mathew Hodson (mhodson)
tags: added: regression-update
tags: added: xenial
To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.