| 2019-05-07 15:31:40 |
Larry Irwin |
bug |
|
|
added bug |
| 2019-05-09 16:01:16 |
Robie Basak |
bug watch added |
|
https://github.com/hercules-team/augeas/issues/569 |
|
| 2019-05-09 16:01:16 |
Robie Basak |
bug task added |
|
augeas |
|
| 2019-05-09 16:01:24 |
Robie Basak |
augeas (Ubuntu): status |
New |
Triaged |
|
| 2019-05-09 16:01:29 |
Robie Basak |
augeas (Ubuntu): importance |
Undecided |
High |
|
| 2019-05-09 16:01:38 |
Robie Basak |
bug |
|
|
added subscriber Ubuntu Server |
| 2019-05-09 16:03:25 |
Robie Basak |
nominated for series |
|
Ubuntu Cosmic |
|
| 2019-05-09 16:03:25 |
Robie Basak |
bug task added |
|
augeas (Ubuntu Cosmic) |
|
| 2019-05-09 16:03:25 |
Robie Basak |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-05-09 16:03:25 |
Robie Basak |
bug task added |
|
augeas (Ubuntu Bionic) |
|
| 2019-05-09 16:03:31 |
Robie Basak |
augeas (Ubuntu): status |
Triaged |
Fix Released |
|
| 2019-05-09 16:03:33 |
Robie Basak |
augeas (Ubuntu Bionic): status |
New |
Triaged |
|
| 2019-05-09 16:03:35 |
Robie Basak |
augeas (Ubuntu Cosmic): status |
New |
Triaged |
|
| 2019-05-09 16:03:36 |
Robie Basak |
augeas (Ubuntu Bionic): importance |
Undecided |
High |
|
| 2019-05-09 16:03:38 |
Robie Basak |
augeas (Ubuntu Cosmic): importance |
Undecided |
High |
|
| 2019-05-09 20:25:57 |
Bug Watch Updater |
augeas: status |
Unknown |
Fix Released |
|
| 2019-05-19 05:34:38 |
Robie Basak |
tags |
|
bitesize |
|
| 2020-05-05 20:38:20 |
Bryce Harrington |
augeas (Ubuntu Cosmic): status |
Triaged |
Won't Fix |
|
| 2021-09-17 10:36:23 |
Miriam España Acebal |
augeas (Ubuntu Bionic): assignee |
|
Miriam España Acebal (mirespace) |
|
| 2021-09-21 10:14:26 |
Miriam España Acebal |
augeas (Ubuntu Bionic): status |
Triaged |
In Progress |
|
| 2021-09-21 10:59:12 |
Miriam España Acebal |
description |
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported. |
[SRU]
[Impact]
The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- Copy the attached conf file here to /etc/apache2/sites-available and make the site available to apache (a2ensite memorybomb).
- Mime module has to be enabled as well (check it with a2query -m | grep mime). If not, enabling it with a2enmod mime.
- Reload apache2 configuration ( systemctl reload apache2 ). If something wrong here, maybe a non-graphical web browser like lynx is need to be installed.
- Open htop in a different terminal
- Run augtool -I httpd.aug
- In augtool prompt use: match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- Check in htop that augtool is consuming:
- Before fixing the issue: near 2238MB
- After fixing the issue (updated packages needed are augeas-lenses, augeas-tools ans libaugeas0): near 179 MB
PPA with built package with fix for testing: ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
This patch is cherry picked from upstream, so the bug is corrected in following augeas package versions.
[Other Info]
Fixed upstream in version 1.11.0, commit id : bbf31f719db54916993be9042254f6d77b61cb13
Commit description: Replace pure function invocations in path expressions with their result.
In path expressions, we generally need to evaluate functions against
every node that we consider for the result set. For example, in the path
expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
function was evaluated against every entry in /etc/hosts. Evaluating that
function requires the construction and compilation of a new regexp.
Because of how memory is managed during evaluation of path expressions, the memory used by all these copies of the same regexp is only freed after we are done evaluating the path expression. This causes unacceptable memory usage in large files (see hercules-team#569).
To avoid these issues, we now distinguish between pure and impure functions in the path expression interpreter. When we encounter a pure function, we change the AST for the path expression so that the function invocation is replaced with the result of invoking the function. With the example above, that means we only construct and compile the regexp '127\\.' once, regardless of how many nodes it gets checked against. That leads to a dramatic reduction in the memory required to evaluate path expressions with such constructs against large files.
Fixes hercules-team#569
Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported. |
|
| 2021-09-21 11:06:08 |
Miriam España Acebal |
bug watch added |
|
https://github.com/certbot/certbot/issues/7035 |
|
| 2021-09-21 11:06:08 |
Miriam España Acebal |
attachment added |
|
Conf site file for test the fixing https://bugs.launchpad.net/ubuntu/+source/augeas/+bug/1828074/+attachment/5526639/+files/memorybomb.conf |
|
| 2021-09-21 11:36:43 |
Miriam España Acebal |
description |
[SRU]
[Impact]
The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- Copy the attached conf file here to /etc/apache2/sites-available and make the site available to apache (a2ensite memorybomb).
- Mime module has to be enabled as well (check it with a2query -m | grep mime). If not, enabling it with a2enmod mime.
- Reload apache2 configuration ( systemctl reload apache2 ). If something wrong here, maybe a non-graphical web browser like lynx is need to be installed.
- Open htop in a different terminal
- Run augtool -I httpd.aug
- In augtool prompt use: match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- Check in htop that augtool is consuming:
- Before fixing the issue: near 2238MB
- After fixing the issue (updated packages needed are augeas-lenses, augeas-tools ans libaugeas0): near 179 MB
PPA with built package with fix for testing: ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
This patch is cherry picked from upstream, so the bug is corrected in following augeas package versions.
[Other Info]
Fixed upstream in version 1.11.0, commit id : bbf31f719db54916993be9042254f6d77b61cb13
Commit description: Replace pure function invocations in path expressions with their result.
In path expressions, we generally need to evaluate functions against
every node that we consider for the result set. For example, in the path
expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
function was evaluated against every entry in /etc/hosts. Evaluating that
function requires the construction and compilation of a new regexp.
Because of how memory is managed during evaluation of path expressions, the memory used by all these copies of the same regexp is only freed after we are done evaluating the path expression. This causes unacceptable memory usage in large files (see hercules-team#569).
To avoid these issues, we now distinguish between pure and impure functions in the path expression interpreter. When we encounter a pure function, we change the AST for the path expression so that the function invocation is replaced with the result of invoking the function. With the example above, that means we only construct and compile the regexp '127\\.' once, regardless of how many nodes it gets checked against. That leads to a dramatic reduction in the memory required to evaluate path expressions with such constructs against large files.
Fixes hercules-team#569
Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported. |
[SRU]
[Impact]
The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- Copy the attached conf file here to /etc/apache2/sites-available and make the site available to apache (a2ensite memorybomb).
- Mime module has to be enabled as well (check it with a2query -m | grep mime). If not, enabling it with a2enmod mime.
- Reload apache2 configuration ( systemctl reload apache2 ). If something wrong here, maybe a non-graphical web browser like lynx is need to be installed.
- Open htop in a different terminal
- Run augtool -I httpd.aug
- In augtool prompt use: match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- Check in htop that augtool is consuming:
- Before fixing the issue: near 2238MB
- After fixing the issue (updated packages needed are augeas-lenses, augeas-tools and libaugeas0): near 179 MB
PPA with built package with fix for testing: ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
This patch is cherry picked from upstream, so the bug is corrected in following augeas package versions.
[Other Info]
Fixed upstream in version 1.11.0, commit id : bbf31f719db54916993be9042254f6d77b61cb13
Commit description: Replace pure function invocations in path expressions with their result.
In path expressions, we generally need to evaluate functions against
every node that we consider for the result set. For example, in the path
expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
function was evaluated against every entry in /etc/hosts. Evaluating that
function requires the construction and compilation of a new regexp.
Because of how memory is managed during evaluation of path expressions, the memory used by all these copies of the same regexp is only freed after we are done evaluating the path expression. This causes unacceptable memory usage in large files (see hercules-team#569).
To avoid these issues, we now distinguish between pure and impure functions in the path expression interpreter. When we encounter a pure function, we change the AST for the path expression so that the function invocation is replaced with the result of invoking the function. With the example above, that means we only construct and compile the regexp '127\\.' once, regardless of how many nodes it gets checked against. That leads to a dramatic reduction in the memory required to evaluate path expressions with such constructs against large files.
Fixes hercules-team#569
Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported. |
|
| 2021-09-21 11:40:40 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~mirespace/ubuntu/+source/augeas/+git/augeas/+merge/408941 |
|
| 2021-09-29 21:16:52 |
Miriam España Acebal |
description |
[SRU]
[Impact]
The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- Copy the attached conf file here to /etc/apache2/sites-available and make the site available to apache (a2ensite memorybomb).
- Mime module has to be enabled as well (check it with a2query -m | grep mime). If not, enabling it with a2enmod mime.
- Reload apache2 configuration ( systemctl reload apache2 ). If something wrong here, maybe a non-graphical web browser like lynx is need to be installed.
- Open htop in a different terminal
- Run augtool -I httpd.aug
- In augtool prompt use: match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- Check in htop that augtool is consuming:
- Before fixing the issue: near 2238MB
- After fixing the issue (updated packages needed are augeas-lenses, augeas-tools and libaugeas0): near 179 MB
PPA with built package with fix for testing: ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
This patch is cherry picked from upstream, so the bug is corrected in following augeas package versions.
[Other Info]
Fixed upstream in version 1.11.0, commit id : bbf31f719db54916993be9042254f6d77b61cb13
Commit description: Replace pure function invocations in path expressions with their result.
In path expressions, we generally need to evaluate functions against
every node that we consider for the result set. For example, in the path
expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
function was evaluated against every entry in /etc/hosts. Evaluating that
function requires the construction and compilation of a new regexp.
Because of how memory is managed during evaluation of path expressions, the memory used by all these copies of the same regexp is only freed after we are done evaluating the path expression. This causes unacceptable memory usage in large files (see hercules-team#569).
To avoid these issues, we now distinguish between pure and impure functions in the path expression interpreter. When we encounter a pure function, we change the AST for the path expression so that the function invocation is replaced with the result of invoking the function. With the example above, that means we only construct and compile the regexp '127\\.' once, regardless of how many nodes it gets checked against. That leads to a dramatic reduction in the memory required to evaluate path expressions with such constructs against large files.
Fixes hercules-team#569
Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported. |
[SRU]
[Impact]
The use of memory grows as a site conf file enlarges when using augeas.
[Test Plan]
- Copy the attached conf file here to /etc/apache2/sites-available and make the site available to apache (a2ensite memorybomb).
- Mime module has to be enabled as well (check it with a2query -m | grep mime). If not, enabling it with a2enmod mime.
- Reload apache2 configuration ( systemctl reload apache2 ). If something wrong here, maybe a non-graphical web browser like lynx is need to be installed.
- Open htop in a different terminal
- Run augtool -I httpd.aug
- In augtool prompt use: match /files/etc/apache2/sites-available/*[label()=~regexp('.*\\.conf')]//*[self::directive=~regexp('([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee])|([Ii][Nn][Cc][Ll][Uu][Dd][Ee][Oo][Pp][Tt][Ii][Oo][Nn][Aa][Ll])')]
- Check in htop that augtool is consuming:
- Before fixing the issue: near 2238MB
- After fixing the issue (updated packages needed are augeas-lenses, augeas-tools and libaugeas0): near 179 MB
PPA with built package with fix for testing: ppa:mirespace/sru-lp-1828074-bionic-augeas-large-http-files
[Where problems could occur]
This patch is cherry picked from upstream, so the bug is corrected in following augeas package versions and no other conflicts have arisen with this change.
Also, the change itself acts as a cache of evaluated expressions (substituting the abstract syntax tree in memory for the value it produces), so the way expressions are evaluated is the same before and after the change and no other part of augeas has to be affected by this.
[Other Info]
Fixed upstream in version 1.11.0, commit id : bbf31f719db54916993be9042254f6d77b61cb13
Commit description: Replace pure function invocations in path expressions with their result.
In path expressions, we generally need to evaluate functions against
every node that we consider for the result set. For example, in the path
expression /files/etc/hosts/*[ipaddr =~ regexp('127\\.')], the regexp
function was evaluated against every entry in /etc/hosts. Evaluating that
function requires the construction and compilation of a new regexp.
Because of how memory is managed during evaluation of path expressions, the memory used by all these copies of the same regexp is only freed after we are done evaluating the path expression. This causes unacceptable memory usage in large files (see hercules-team#569).
To avoid these issues, we now distinguish between pure and impure functions in the path expression interpreter. When we encounter a pure function, we change the AST for the path expression so that the function invocation is replaced with the result of invoking the function. With the example above, that means we only construct and compile the regexp '127\\.' once, regardless of how many nodes it gets checked against. That leads to a dramatic reduction in the memory required to evaluate path expressions with such constructs against large files.
Fixes hercules-team#569
Bug on upstream: https://github.com/hercules-team/augeas/issues/569
[Original Report]
This bug in libaugeas was fixed in version 1.11.0, but is not in the 18.04.2 repo.
See: https://github.com/hercules-team/augeas/issues/569
I found it using certbot and started a bug report there.
See: https://github.com/certbot/certbot/issues/7035
Which led me here to see if the newer version could be back-ported. |
|
| 2021-10-08 05:48:20 |
Miriam España Acebal |
bug |
|
|
added subscriber Miriam España Acebal |
| 2021-10-13 14:08:34 |
Robie Basak |
augeas (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2021-10-13 14:08:36 |
Robie Basak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2021-10-13 14:08:37 |
Robie Basak |
bug |
|
|
added subscriber SRU Verification |
| 2021-10-13 14:08:40 |
Robie Basak |
tags |
bitesize |
bitesize verification-needed verification-needed-bionic |
|
| 2021-10-18 13:09:48 |
Miriam España Acebal |
attachment added |
|
Image for bionic verification https://bugs.launchpad.net/ubuntu/+source/augeas/+bug/1828074/+attachment/5533838/+files/Before-and-after-fix-augeas-lp1828074.png |
|
| 2021-10-18 13:12:47 |
Miriam España Acebal |
tags |
bitesize verification-needed verification-needed-bionic |
bitesize verification-done verification-done-bionic |
|
| 2021-10-20 20:52:49 |
Launchpad Janitor |
augeas (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2021-10-20 20:52:57 |
Brian Murray |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
