| 2014-07-22 20:24:56 |
Benjamin Kaduk |
bug |
|
|
added bug |
| 2014-07-24 01:04:13 |
Anders Kaseorg |
bug |
|
|
added subscriber Anders Kaseorg |
| 2014-07-29 21:45:49 |
Taylor Yu |
krb5 (Ubuntu): status |
New |
Confirmed |
|
| 2014-07-29 22:54:52 |
Anders Kaseorg |
tags |
|
regression-release testcase trusty utopic |
|
| 2014-07-30 13:05:52 |
Anders Kaseorg |
bug task added |
|
gcc-4.8 (Ubuntu) |
|
| 2014-07-30 13:06:23 |
Anders Kaseorg |
attachment added |
|
Reduced test case for GCC 4.8 bug https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1347147/+attachment/4165776/+files/bug.c |
|
| 2014-07-30 13:22:09 |
Anders Kaseorg |
bug watch added |
|
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=61964 |
|
| 2014-07-30 13:22:09 |
Anders Kaseorg |
bug task added |
|
gcc |
|
| 2014-07-30 13:22:43 |
Anders Kaseorg |
bug watch added |
|
http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 |
|
| 2014-07-30 13:22:43 |
Anders Kaseorg |
bug task added |
|
kerberos |
|
| 2014-07-30 21:46:00 |
Robie Basak |
bug |
|
|
added subscriber Robie Basak |
| 2014-07-30 21:46:04 |
Robie Basak |
krb5 (Ubuntu): status |
Confirmed |
Triaged |
|
| 2014-07-31 01:37:48 |
Launchpad Janitor |
branch linked |
|
lp:~hartmans/ubuntu/trusty/krb5/gss-infinite-loop |
|
| 2014-07-31 15:33:55 |
Taylor Yu |
summary |
krb5 database propagation enters infinite loop |
krb5 database operations enter infinite loop |
|
| 2014-07-31 15:53:44 |
Launchpad Janitor |
gcc-4.8 (Ubuntu): status |
New |
Confirmed |
|
| 2014-07-31 16:03:58 |
Sam Hartman |
description |
In some conditions, propagating a kerberos database to a slave KDC server can stall.
This is due to a misoptimization by gcc 4.8 of the CIRCLEQ famliy of macros, apparently due to overzealous strict aliasing deductions.
One case of this stall is reported at http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html (and the rest of the thread), and there is an entry in the upstream bugtracker at http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 .
gcc 4.9 (as used in Debian unstable at present) is not believed to induce this problem. Upstream has patched their code to use the TAILQ family of macros instead, as a workaround, but that workaround has not yet appeared in an upstream release: https://github.com/krb5/krb5/commit/26d8744129
Because of the different compiler versions used on Debian and Ubuntu, I am filing this as an Ubuntu-specific bug. |
In some conditions, propagating a kerberos database to a slave KDC server or performing other database operations can stall. As we've investigated the issue, it looks like a database with more than a few hundred principals is very likely to run into this issue.
This is due to a misoptimization by gcc 4.8 of the CIRCLEQ famliy of macros, apparently due to overzealous strict aliasing deductions.
One case of this stall is reported at http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html (and the rest of the thread), and there is an entry in the upstream bugtracker at http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 .
gcc 4.9 (as used in Debian unstable at present) is not believed to induce this problem. Upstream has patched their code to use the TAILQ family of macros instead, as a workaround, but that workaround has not yet appeared in an upstream release: https://github.com/krb5/krb5/commit/26d8744129
A branch is linked including this upstream work around and two other patches to bugs already nominated for trusty applied to the krb5 in trusty. We believe the impact is significant because this is likely to be a problem for sites with a large database running trusty. The regression potential is very small. The upstream work around changes from one family of queue macros that are stable and well-tested to another.
For utopic, the simplest fix is to rebuild krb5 with the compiler currently in utopic. An alternative is to request that the Debian maintainers (both monitoring this bug for such a request) upload the upstream work around to Debian and sync that. You could do an ubuntu-specific upload but it seems undesirable to introduce a change between Ubuntu and Debian when all the right parties are happy to avoid it.
Because of the different compiler versions used on Debian and Ubuntu, I am filing this as an Ubuntu-specific bug. |
|
| 2014-07-31 16:06:08 |
Taylor Yu |
description |
In some conditions, propagating a kerberos database to a slave KDC server or performing other database operations can stall. As we've investigated the issue, it looks like a database with more than a few hundred principals is very likely to run into this issue.
This is due to a misoptimization by gcc 4.8 of the CIRCLEQ famliy of macros, apparently due to overzealous strict aliasing deductions.
One case of this stall is reported at http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html (and the rest of the thread), and there is an entry in the upstream bugtracker at http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 .
gcc 4.9 (as used in Debian unstable at present) is not believed to induce this problem. Upstream has patched their code to use the TAILQ family of macros instead, as a workaround, but that workaround has not yet appeared in an upstream release: https://github.com/krb5/krb5/commit/26d8744129
A branch is linked including this upstream work around and two other patches to bugs already nominated for trusty applied to the krb5 in trusty. We believe the impact is significant because this is likely to be a problem for sites with a large database running trusty. The regression potential is very small. The upstream work around changes from one family of queue macros that are stable and well-tested to another.
For utopic, the simplest fix is to rebuild krb5 with the compiler currently in utopic. An alternative is to request that the Debian maintainers (both monitoring this bug for such a request) upload the upstream work around to Debian and sync that. You could do an ubuntu-specific upload but it seems undesirable to introduce a change between Ubuntu and Debian when all the right parties are happy to avoid it.
Because of the different compiler versions used on Debian and Ubuntu, I am filing this as an Ubuntu-specific bug. |
[Impact]
On krb5 KDC databases with more than a few hundred principals, operations can enter an infinite loop in the database library. This affects both read and write operations. If operators are fortunate, they will encounter this bug while testing a migration. If they are not so fortunate, they will encounter this bug in a production KDC when the number of principals crosses the threshold where this bug manifests, resulting in a service outage and possible database corruption. Probably the only way to restore service in that situation is to install a patched KDC or to downgrade to an unaffected version.
Both Trusty and Utopic amd64 have been verified to have this issue.
One concrete reported example is an invocation of kdb5_util load (as part of a slave KDC propagation) spinning:
http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html
Additional failure modes are likely
The proposed fix at https://launchpad.net/~hartmans/+archive/ubuntu/ubuntu-fixes
works around a compiler optimizer bug in the gcc-4.8 series, which incorrectly deduces that a strict aliasing violation has occurred and miscompiles part of the bundled libdb2 library that the KDC database back end depends upon. The miscompilation causes a data structure to contain an inappropriate cycle, which leads to an infinite loop when the structure is traversed.
[Test Case]
apt-get install krb5-kdc krb5-admin-server
kdb5_util -W -r T create -s
awk 'BEGIN{ for (i = 0; i < 1024; i++) { printf("ank -randkey a%06d\n", i) } }' /dev/null | kadmin.local -r T
(Enter any password for the master key when requested.)
On platforms with this issue, kadmin.local spins consuming 100% CPU after a few hundred principals have been created. (This is "a000762" on two examples.)
To clean up,
rm /etc/krb5kdc/principal*
or
krb5kdc -r T destroy
but the latter can possibly enter the same infinite loop.
[Regression Potential]
Negligible.
It is theoretically possible that our upstream workaround, which involves using TAILQ macros instead of CIRCLEQ macros in the bundled libdb2 that backs the KDC database, will have some as-yet undiscovered bugs or compiler interactions with consequences worse than this current issue. I think this is rather unlikely.
The patched libdb2 passes both the extensive libdb2 test suite and the rest of the krb5 test suite. Prior to patching, compiling krb5 with an affected gcc would cause the krb5 test suite to stall when it reached the libdb2 test suite. (The test suite stall is how we became aware of the gcc optimizer bug.)
The BSD TAILQ macros are generally considered to be safer than the CIRCLEQ macros, and the various open-source BSD derivatives have made the corresponding change to their libdb sources years ago, with no reported ill effects that I can see.
Original report from Ben Kaduk:
==========
In some conditions, propagating a kerberos database to a slave KDC server can stall.
This is due to a misoptimization by gcc 4.8 of the CIRCLEQ famliy of macros, apparently due to overzealous strict aliasing deductions.
One case of this stall is reported at http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html (and the rest of the thread), and there is an entry in the upstream bugtracker at http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 .
gcc 4.9 (as used in Debian unstable at present) is not believed to induce this problem. Upstream has patched their code to use the TAILQ family of macros instead, as a workaround, but that workaround has not yet appeared in an upstream release: https://github.com/krb5/krb5/commit/26d8744129
Because of the different compiler versions used on Debian and Ubuntu, I am filing this as an Ubuntu-specific bug. |
|
| 2014-07-31 16:24:52 |
Taylor Yu |
description |
[Impact]
On krb5 KDC databases with more than a few hundred principals, operations can enter an infinite loop in the database library. This affects both read and write operations. If operators are fortunate, they will encounter this bug while testing a migration. If they are not so fortunate, they will encounter this bug in a production KDC when the number of principals crosses the threshold where this bug manifests, resulting in a service outage and possible database corruption. Probably the only way to restore service in that situation is to install a patched KDC or to downgrade to an unaffected version.
Both Trusty and Utopic amd64 have been verified to have this issue.
One concrete reported example is an invocation of kdb5_util load (as part of a slave KDC propagation) spinning:
http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html
Additional failure modes are likely
The proposed fix at https://launchpad.net/~hartmans/+archive/ubuntu/ubuntu-fixes
works around a compiler optimizer bug in the gcc-4.8 series, which incorrectly deduces that a strict aliasing violation has occurred and miscompiles part of the bundled libdb2 library that the KDC database back end depends upon. The miscompilation causes a data structure to contain an inappropriate cycle, which leads to an infinite loop when the structure is traversed.
[Test Case]
apt-get install krb5-kdc krb5-admin-server
kdb5_util -W -r T create -s
awk 'BEGIN{ for (i = 0; i < 1024; i++) { printf("ank -randkey a%06d\n", i) } }' /dev/null | kadmin.local -r T
(Enter any password for the master key when requested.)
On platforms with this issue, kadmin.local spins consuming 100% CPU after a few hundred principals have been created. (This is "a000762" on two examples.)
To clean up,
rm /etc/krb5kdc/principal*
or
krb5kdc -r T destroy
but the latter can possibly enter the same infinite loop.
[Regression Potential]
Negligible.
It is theoretically possible that our upstream workaround, which involves using TAILQ macros instead of CIRCLEQ macros in the bundled libdb2 that backs the KDC database, will have some as-yet undiscovered bugs or compiler interactions with consequences worse than this current issue. I think this is rather unlikely.
The patched libdb2 passes both the extensive libdb2 test suite and the rest of the krb5 test suite. Prior to patching, compiling krb5 with an affected gcc would cause the krb5 test suite to stall when it reached the libdb2 test suite. (The test suite stall is how we became aware of the gcc optimizer bug.)
The BSD TAILQ macros are generally considered to be safer than the CIRCLEQ macros, and the various open-source BSD derivatives have made the corresponding change to their libdb sources years ago, with no reported ill effects that I can see.
Original report from Ben Kaduk:
==========
In some conditions, propagating a kerberos database to a slave KDC server can stall.
This is due to a misoptimization by gcc 4.8 of the CIRCLEQ famliy of macros, apparently due to overzealous strict aliasing deductions.
One case of this stall is reported at http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html (and the rest of the thread), and there is an entry in the upstream bugtracker at http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 .
gcc 4.9 (as used in Debian unstable at present) is not believed to induce this problem. Upstream has patched their code to use the TAILQ family of macros instead, as a workaround, but that workaround has not yet appeared in an upstream release: https://github.com/krb5/krb5/commit/26d8744129
Because of the different compiler versions used on Debian and Ubuntu, I am filing this as an Ubuntu-specific bug. |
[Impact]
On krb5 KDC databases with more than a few hundred principals, operations can enter an infinite loop in the database library. This affects both read and write operations. If operators are fortunate, they will encounter this bug while testing a migration. If they are not so fortunate, they will encounter this bug in a production KDC when the number of principals crosses the threshold where this bug manifests, resulting in a service outage and possible database corruption. Probably the only way to restore service in that situation is to install a patched KDC or to downgrade to an unaffected version.
Both Trusty and Utopic amd64 have been verified to have this issue.
One concrete reported example is an invocation of kdb5_util load (as part of a slave KDC propagation) spinning:
http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html
Additional failure modes are likely
A branch is linked including the upstream work around for this bug, along with two other patches to bugs already nominated for trusty applied to the krb5 in trusty.
For utopic, the simplest fix is to rebuild krb5 with the compiler currently in utopic. An alternative is to request that the Debian maintainers (both monitoring this bug for such a request) upload the upstream work around to Debian and sync that. You could do an ubuntu-specific upload but it seems undesirable to introduce a change between Ubuntu and Debian when all the right parties are happy to avoid it.
The upstream patch works around a compiler optimizer bug in the gcc-4.8 series, which incorrectly deduces that a strict aliasing violation has occurred and miscompiles part of the bundled libdb2 library that the KDC database back end depends upon. The miscompilation causes a data structure to contain an inappropriate cycle, which leads to an infinite loop when the structure is traversed.
[Test Case]
apt-get install krb5-kdc krb5-admin-server
kdb5_util -W -r T create -s
awk 'BEGIN{ for (i = 0; i < 1024; i++) { printf("ank -randkey a%06d\n", i) } }' /dev/null | kadmin.local -r T
(Enter any password for the master key when requested.)
On platforms with this issue, kadmin.local spins consuming 100% CPU after a few hundred principals have been created. (This is "a000762" on two examples.)
To clean up,
rm /etc/krb5kdc/principal*
or
krb5kdc -r T destroy
but the latter can possibly enter the same infinite loop.
[Regression Potential]
Negligible.
It is theoretically possible that our upstream workaround, which involves using TAILQ macros instead of CIRCLEQ macros in the bundled libdb2 that backs the KDC database, will have some as-yet undiscovered bugs or compiler interactions with consequences worse than this current issue. I think this is rather unlikely.
The patched libdb2 passes both the extensive libdb2 test suite and the rest of the krb5 test suite. Prior to patching, compiling krb5 with an affected gcc would cause the krb5 test suite to stall when it reached the libdb2 test suite. (The test suite stall is how we became aware of the gcc optimizer bug.)
The BSD TAILQ macros are generally considered to be safer than the CIRCLEQ macros, and the various open-source BSD derivatives have made the corresponding change to their libdb sources years ago, with no reported ill effects that I can see.
Original report from Ben Kaduk:
==========
In some conditions, propagating a kerberos database to a slave KDC server can stall.
This is due to a misoptimization by gcc 4.8 of the CIRCLEQ famliy of macros, apparently due to overzealous strict aliasing deductions.
One case of this stall is reported at http://mailman.mit.edu/pipermail/kerberos/2014-July/020007.html (and the rest of the thread), and there is an entry in the upstream bugtracker at http://krbdev.mit.edu/rt/Ticket/Display.html?id=7860 .
gcc 4.9 (as used in Debian unstable at present) is not believed to induce this problem. Upstream has patched their code to use the TAILQ family of macros instead, as a workaround, but that workaround has not yet appeared in an upstream release: https://github.com/krb5/krb5/commit/26d8744129
Because of the different compiler versions used on Debian and Ubuntu, I am filing this as an Ubuntu-specific bug. |
|
| 2014-07-31 18:38:59 |
Gunnar Hjalmarsson |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2014-07-31 19:12:51 |
Sam Hartman |
attachment added |
|
debdiff between current trusty and linked branch https://bugs.launchpad.net/gcc/+bug/1347147/+attachment/4166949/+files/krb5-trusty-stable.debdiff |
|
| 2014-07-31 19:35:50 |
Thomas Ward |
nominated for series |
|
Ubuntu Trusty |
|
| 2014-08-01 04:46:28 |
Martin Pitt |
bug task added |
|
krb5 (Ubuntu Trusty) |
|
| 2014-08-01 04:46:28 |
Martin Pitt |
bug task added |
|
gcc-4.8 (Ubuntu Trusty) |
|
| 2014-08-01 08:09:43 |
Anders Kaseorg |
bug task added |
|
gcc-4.9 (Ubuntu) |
|
| 2014-08-01 08:20:57 |
Anders Kaseorg |
bug task deleted |
gcc-4.9 (Ubuntu Trusty) |
|
|
| 2014-08-01 21:19:34 |
Anders Kaseorg |
cve linked |
|
2014-5044 |
|
| 2014-08-01 21:19:34 |
Anders Kaseorg |
gcc-4.8 (Ubuntu): status |
Confirmed |
Fix Committed |
|
| 2014-08-01 22:26:34 |
Launchpad Janitor |
branch linked |
|
lp:debian/gcc-4.8 |
|
| 2014-08-02 01:17:19 |
Launchpad Janitor |
branch linked |
|
lp:ubuntu/utopic-proposed/gcc-4.9 |
|
| 2014-08-02 02:02:35 |
Launchpad Janitor |
gcc-4.8 (Ubuntu): status |
Fix Committed |
Fix Released |
|
| 2014-08-02 10:28:51 |
Launchpad Janitor |
gcc-4.9 (Ubuntu): status |
New |
Fix Released |
|
| 2014-08-02 23:41:14 |
Robie Basak |
krb5 (Ubuntu): importance |
Undecided |
High |
|
| 2014-08-02 23:41:22 |
Robie Basak |
krb5 (Ubuntu Trusty): importance |
Undecided |
High |
|
| 2014-08-02 23:41:28 |
Robie Basak |
krb5 (Ubuntu Trusty): status |
New |
Triaged |
|
| 2014-08-04 18:08:43 |
Bug Watch Updater |
gcc: status |
Unknown |
Fix Released |
|
| 2014-08-04 18:08:43 |
Bug Watch Updater |
gcc: importance |
Unknown |
High |
|
| 2014-08-04 18:51:32 |
Anders Kaseorg |
cve linked |
|
2014-4341 |
|
| 2014-08-04 18:51:32 |
Anders Kaseorg |
cve linked |
|
2014-4342 |
|
| 2014-08-04 18:51:32 |
Anders Kaseorg |
cve linked |
|
2014-4343 |
|
| 2014-08-04 18:51:32 |
Anders Kaseorg |
cve linked |
|
2014-4344 |
|
| 2014-08-04 18:51:32 |
Anders Kaseorg |
krb5 (Ubuntu): status |
Triaged |
Fix Committed |
|
| 2014-08-04 19:05:49 |
Launchpad Janitor |
krb5 (Ubuntu): status |
Fix Committed |
Fix Released |
|
| 2014-08-09 02:47:12 |
Launchpad Janitor |
gcc-4.8 (Ubuntu Trusty): status |
New |
Confirmed |
|
| 2014-08-09 02:57:38 |
Anders Kaseorg |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2014-08-09 22:23:03 |
Launchpad Janitor |
branch linked |
|
lp:debian/gcc-4.9 |
|
| 2014-08-12 11:36:44 |
Sam Hartman |
attachment removed |
debdiff between current trusty and linked branch https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1347147/+attachment/4166949/+files/krb5-trusty-stable.debdiff |
|
|
| 2014-08-12 11:38:40 |
Sam Hartman |
attachment added |
|
Debdiff of lp:~hartmans/ubuntu/trusty/krb5/gss-infinite-loop https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1347147/+attachment/4175346/+files/krb5_1.12%2Bdfsg-2ubuntu5.debdiff |
|
| 2014-09-21 04:23:35 |
Launchpad Janitor |
branch linked |
|
lp:debian/gnat-4.9 |
|
| 2014-09-29 10:47:45 |
Iain Lane |
krb5 (Ubuntu Trusty): status |
Triaged |
In Progress |
|
| 2014-09-29 10:47:54 |
Iain Lane |
krb5 (Ubuntu Trusty): assignee |
|
Sam Hartman (hartmans) |
|
| 2014-09-29 18:58:21 |
Sebastien Bacher |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2014-10-02 18:13:49 |
Brian Murray |
krb5 (Ubuntu Trusty): status |
In Progress |
Fix Committed |
|
| 2014-10-02 18:13:53 |
Brian Murray |
bug |
|
|
added subscriber SRU Verification |
| 2014-10-02 18:14:03 |
Brian Murray |
tags |
regression-release testcase trusty utopic |
regression-release testcase trusty utopic verification-needed |
|
| 2014-10-02 23:05:07 |
Taylor Yu |
tags |
regression-release testcase trusty utopic verification-needed |
regression-release testcase trusty utopic verification-done |
|
| 2014-12-03 07:02:10 |
Mathew Hodson |
branch linked |
|
lp:ubuntu/trusty-proposed/krb5 |
|
| 2014-12-03 07:05:18 |
Mathew Hodson |
branch unlinked |
lp:debian/gnat-4.9 |
|
|
| 2014-12-03 07:06:41 |
Mathew Hodson |
branch linked |
|
lp:ubuntu/utopic-proposed/gcc-4.8 |
|
| 2015-02-09 13:35:34 |
Launchpad Janitor |
krb5 (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
| 2015-02-09 13:36:01 |
Scott Kitterman |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2015-05-14 15:27:13 |
Steve Langasek |
gcc-4.8 (Ubuntu Trusty): status |
Confirmed |
Fix Committed |
|
| 2015-05-14 15:27:17 |
Steve Langasek |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2015-05-14 15:27:22 |
Steve Langasek |
tags |
regression-release testcase trusty utopic verification-done |
regression-release testcase trusty utopic |
|
| 2015-05-14 15:27:23 |
Steve Langasek |
tags |
regression-release testcase trusty utopic |
regression-release testcase trusty utopic verification-needed |
|
| 2015-06-10 09:20:13 |
Matthias Klose |
tags |
regression-release testcase trusty utopic verification-needed |
regression-release testcase trusty utopic verification-done |
|
| 2015-06-18 06:23:37 |
Launchpad Janitor |
gcc-4.8 (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
| 2015-11-24 19:02:11 |
Steve Langasek |
tags |
regression-release testcase trusty utopic verification-done |
regression-release testcase trusty utopic |
|
| 2015-11-24 19:02:12 |
Steve Langasek |
tags |
regression-release testcase trusty utopic |
regression-release testcase trusty utopic verification-needed |
|
| 2016-01-06 19:16:22 |
Steve Langasek |
tags |
regression-release testcase trusty utopic verification-needed |
regression-release testcase trusty utopic verification-done |
|
