Memory usage issues with large Apache httpd configuration files

Bug #1828074 reported by Larry Irwin
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fix Released
augeas (Ubuntu)
Fix Released
Fix Released
Miriam España Acebal
Won't Fix

Bug Description



 The use of memory grows as a site conf file enlarges when using augeas.

[Test Plan]

  - Copy the attached conf file here to /etc/apache2/sites-available and make the site available to apache (a2ensite memorybomb).
  - Mime module has to be enabled as well (check it with a2query -m | grep mime). If not, enabling it with a2enmod mime.
  - Reload apache2 configuration ( systemctl reload apache2 ). If something wrong here, maybe a non-graphical web browser like lynx is need to be installed.
  - Open htop in a different terminal
  - Run augtool -I httpd.aug
  - In augtool prompt use: match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
  - Check in htop that augtool is consuming:
    - Before fixing the issue: near 2238MB
    - After fixing the issue (updated packages needed are augeas-lenses, augeas-tools and libaugeas0): near 179 MB

 PPA with built package with fix for testing: ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files

[Where problems could occur]

 This patch is cherry picked from upstream, so the bug is corrected in following augeas package versions and no other conflicts have arisen with this change.

    Also, the change itself acts as a cache of evaluated expressions (substituting the abstract syntax tree in memory for the value it produces), so the way expressions are evaluated is the same before and after the change and no other part of augeas has to be affected by this.

[Other Info]

 Fixed upstream in version 1.11.0, commit id : bbf31f719db54916993be9042254f6d77b61cb13

 Commit description: Replace pure function invocations in path expressions with their result.

 In path expressions, we generally need to evaluate functions against
 every node that we consider for the result set. For example, in the path
 expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
 function was evaluated against every entry in /etc/hosts. Evaluating that
 function requires the construction and compilation of a new regexp.

Because of how memory is managed during evaluation of path expressions, the memory used by all these copies of the same regexp is only freed after we are done evaluating the path expression. This causes unacceptable memory usage in large files (see hercules-team#569).

 To avoid these issues, we now distinguish between pure and impure functions in the path expression interpreter. When we encounter a pure function, we change the AST for the path expression so that the function invocation is replaced with the result of invoking the function. With the example above, that means we only construct and compile the regexp '127\\.' once, regardless of how many nodes it gets checked against. That leads to a dramatic reduction in the memory required to evaluate path expressions with such constructs against large files.

 Fixes hercules-team#569

 Bug on upstream:

[Original Report]

This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
I found it using certbot and started a bug report there.
Which led me here to see if the newer version could be back-ported.

Related branches

Revision history for this message
Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

We can cherry-pick a fix to Ubuntu 18.04. Does the upstream commit apply to the 18.04 Ubuntu packaging cleanly? It would help if someone could prepare the upload please.

Changed in augeas (Ubuntu):
status: New → Triaged
importance: Undecided → High
status: Triaged → Fix Released
Changed in augeas (Ubuntu Bionic):
status: New → Triaged
Changed in augeas (Ubuntu Cosmic):
status: New → Triaged
Changed in augeas (Ubuntu Bionic):
importance: Undecided → High
Changed in augeas (Ubuntu Cosmic):
importance: Undecided → High
Changed in augeas:
status: Unknown → Fix Released
Robie Basak (racb)
tags: added: bitesize
Revision history for this message
Bryce Harrington (bryce) wrote :

Standard support for cosmic has ended as of July 2019

Changed in augeas (Ubuntu Cosmic):
status: Triaged → Won't Fix
Changed in augeas (Ubuntu Bionic):
assignee: nobody → Miriam España Acebal (mirespace)
Changed in augeas (Ubuntu Bionic):
status: Triaged → In Progress
description: updated
Revision history for this message
Miriam España Acebal (mirespace) wrote :

Applied patch for fixing the increase of memory (attached is the conf site file for testing purposes) as suggested by the reporter.

In the upstream's bug description is reported also a segmentation fault caused by very very large files ( but is unplanned when this is going to be fixed upstream. This behaviour also occurs in the package tested here.

description: updated
description: updated
Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello Larry, or anyone else affected,

Accepted augeas into bionic-proposed. The package will build now and be available at in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in augeas (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Miriam España Acebal (mirespace) wrote :

Verification for Bionic

Checking the package is upgradable:

root@test-bug1828074-bionic:/etc/apache2/sites-available# apt list --upgradable -a
Listing... Done
augeas-lenses/bionic 1.10.1-2ubuntu1 all [upgradable from: 1.10.1-2]
augeas-lenses/bionic,now 1.10.1-2 all [installed,upgradable to: 1.10.1-2ubuntu1]

augeas-tools/bionic 1.10.1-2ubuntu1 amd64 [upgradable from: 1.10.1-2]
augeas-tools/bionic,now 1.10.1-2 amd64 [installed,upgradable to: 1.10.1-2ubuntu1]

libaugeas0/bionic 1.10.1-2ubuntu1 amd64 [upgradable from: 1.10.1-2]
libaugeas0/bionic,now 1.10.1-2 amd64 [installed,upgradable to: 1.10.1-2ubuntu1]

If we executes(with memorybomb.conf in sites-available) the following:

augtool> match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
  (no matches)

We can see in the attached image the expenditure in memory before and after the fixing.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package augeas - 1.10.1-2ubuntu1

augeas (1.10.1-2ubuntu1) bionic; urgency=medium

  * d/p/0005-Replace-pure-funcs-calls-in-path-expr-w-result.patch:
    Replace pure function invocations in path
    expressions with their result. (LP: #1828074)
    Thanks to David Lutterkort <email address hidden>.
  * d/rules: Disable gnulib tests due to that tests are not
    testing augeas itself. (Closes: #919662) (LP: #1813566)
    Thanks to Matthias Klose.

 -- Miriam España Acebal <email address hidden> Mon, 20 Sep 2021 11:18:53 +0200

Changed in augeas (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for augeas has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.