Activity log for bug #1752683

Date Who What changed Old value New value Message
2018-03-01 18:26:11 Rafael David Tinoco bug added bug
2018-03-01 18:26:18 Rafael David Tinoco apache2 (Ubuntu): status New In Progress
2018-03-01 18:26:20 Rafael David Tinoco apache2 (Ubuntu): assignee Rafael David Tinoco (inaddy)
2018-03-01 18:26:23 Rafael David Tinoco apache2 (Ubuntu): importance Undecided Medium
2018-03-01 18:27:10 Rafael David Tinoco tags sts
2018-03-01 18:44:18 Fabio Augusto Miranda Martins bug added subscriber Fabio Augusto Miranda Martins
2018-03-02 00:55:48 Rafael David Tinoco nominated for series Ubuntu Trusty
2018-03-02 00:55:48 Rafael David Tinoco nominated for series Ubuntu Xenial
2018-03-02 00:55:48 Rafael David Tinoco nominated for series Ubuntu Bionic
2018-03-02 00:55:48 Rafael David Tinoco nominated for series Ubuntu Artful
2018-03-02 01:05:14 Rafael David Tinoco bug watch added https://bz.apache.org/bugzilla/show_bug.cgi?id=58483
2018-03-02 01:05:14 Rafael David Tinoco bug watch added https://bz.apache.org/bugzilla/show_bug.cgi?id=60296
2018-03-02 01:05:14 Rafael David Tinoco bug watch added https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814980
2018-03-02 02:18:35 Rafael David Tinoco bug task added apache2
2018-03-02 02:25:40 Rafael David Tinoco description Problem summary: apr_rmm_init acts as a relocatable memory management initialization it is used in: mod_auth_digest and util_ldap_cache From the dump was brought to my knowledge, in the following sequence: - util_ldap_compare_node_copy() - util_ald_strdup() - apr_rmm_calloc() - find_block_of_size() Had a "cache->rmm_addr" with no lock at "find_block_of_size()" cache->rmm_addr->lock { type = apr_anylock_none } And an invalid "next" offset (out of rmm->base->firstfree). This rmm_addr was initialized with NULL as a locking mechanism: From apr-utils: apr_rmm_init() if (!lock) { <-- 2nd argument to apr_rmm_init() nulllock.type = apr_anylock_none; <--- found in the dump nulllock.lock.pm = NULL; lock = &nulllock; } From apache: # mod_auth_digest sts = apr_rmm_init(&client_rmm, NULL, /* no lock, we'll do the locking ourselves */ apr_shm_baseaddr_get(client_shm), shmem_size, ctx); # util_ldap_cache result = apr_rmm_init(&st->cache_rmm, NULL, apr_shm_baseaddr_get(st->cache_shm), size, st->pool); It appears that the ldap module chose to use "rmm" for memory allocation, using the shared memory approach, but without explicitly definiting a lock to it. Without it, its up to the caller to guarantee that there are locks for rmm synchronization (just like mod_auth_digest does, using global mutexes). Because of that, there was a race condition in "find_block_of_size" and a call touching "rmm->base->firstfree", possibly "move_block()", in a multi-threaded apache environment, since there were no lock guarantees inside rmm logic (lock was "apr_anylock_none" and the locking calls don't do anything). In find_block_of_size: apr_rmm_off_t next = rmm->base->firstfree; We have: rmm->base->firstfree Decimal:356400 Hex:0x57030 But "next" turned into: Name : next Decimal:8320808657351632189 Hex:0x737973636970653d Causing: struct rmm_block_t *blk = (rmm_block_t*)((char*)rmm->base + next); if (blk->size == size) To segfault. Upstream bugs: https://bz.apache.org/bugzilla/show_bug.cgi?id=58483 https://bz.apache.org/bugzilla/show_bug.cgi?id=60296 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814980#15 [Impact] * Apache users using ldap module might face this if using multiple threads and shared memory activated for apr memory allocator (default in Ubuntu). [Test Case] * Configure apache to use ldap module, for authentication e.g., and wait for the race condition to happen. * Analysis made out of a dump from a production environment. * Bug has been reported multiple times upstream in the past 10 years. [Regression Potential] * ldap module has broken locking mechanism when using apr mem mgmt. * ldap would continue to have broken locking mechanism. * race conditions could still exist. * could could brake ldap module. * patch is upstreamed in next version to be released. [Other Info] ORIGINAL CASE DESCRIPTION: Problem summary: apr_rmm_init acts as a relocatable memory management initialization it is used in: mod_auth_digest and util_ldap_cache From the dump was brought to my knowledge, in the following sequence: - util_ldap_compare_node_copy() - util_ald_strdup() - apr_rmm_calloc() - find_block_of_size() Had a "cache->rmm_addr" with no lock at "find_block_of_size()" cache->rmm_addr->lock { type = apr_anylock_none } And an invalid "next" offset (out of rmm->base->firstfree). This rmm_addr was initialized with NULL as a locking mechanism: From apr-utils: apr_rmm_init()     if (!lock) { <-- 2nd argument to apr_rmm_init()         nulllock.type = apr_anylock_none; <--- found in the dump         nulllock.lock.pm = NULL;         lock = &nulllock;     } From apache: # mod_auth_digest     sts = apr_rmm_init(&client_rmm,                        NULL, /* no lock, we'll do the locking ourselves */                        apr_shm_baseaddr_get(client_shm),                        shmem_size, ctx); # util_ldap_cache         result = apr_rmm_init(&st->cache_rmm, NULL,                               apr_shm_baseaddr_get(st->cache_shm), size,                               st->pool); It appears that the ldap module chose to use "rmm" for memory allocation, using the shared memory approach, but without explicitly definiting a lock to it. Without it, its up to the caller to guarantee that there are locks for rmm synchronization (just like mod_auth_digest does, using global mutexes). Because of that, there was a race condition in "find_block_of_size" and a call touching "rmm->base->firstfree", possibly "move_block()", in a multi-threaded apache environment, since there were no lock guarantees inside rmm logic (lock was "apr_anylock_none" and the locking calls don't do anything). In find_block_of_size:     apr_rmm_off_t next = rmm->base->firstfree; We have:     rmm->base->firstfree  Decimal:356400  Hex:0x57030 But "next" turned into: Name : next  Decimal:8320808657351632189  Hex:0x737973636970653d Causing:         struct rmm_block_t *blk = (rmm_block_t*)((char*)rmm->base + next);         if (blk->size == size) To segfault. Upstream bugs: https://bz.apache.org/bugzilla/show_bug.cgi?id=58483 https://bz.apache.org/bugzilla/show_bug.cgi?id=60296 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814980#15
2018-03-02 02:26:27 Rafael David Tinoco attachment added xenial_apache2_2.4.18-2ubuntu3.6.debdiff https://bugs.launchpad.net/apache2/+bug/1752683/+attachment/5066581/+files/xenial_apache2_2.4.18-2ubuntu3.6.debdiff
2018-03-02 02:27:04 Rafael David Tinoco attachment added trusty_apache2_2.4.7-1ubuntu4.19.debdiff https://bugs.launchpad.net/apache2/+bug/1752683/+attachment/5066582/+files/trusty_apache2_2.4.7-1ubuntu4.19.debdiff
2018-03-02 02:27:23 Rafael David Tinoco attachment added artful_apache2_2.4.27-2ubuntu4.debdiff https://bugs.launchpad.net/apache2/+bug/1752683/+attachment/5066583/+files/artful_apache2_2.4.27-2ubuntu4.debdiff
2018-03-02 02:27:43 Rafael David Tinoco attachment added bionic_apache2_2.4.29-1ubuntu4.debdiff https://bugs.launchpad.net/apache2/+bug/1752683/+attachment/5066584/+files/bionic_apache2_2.4.29-1ubuntu4.debdiff
2018-03-02 09:56:45 Bug Watch Updater apache2: status Unknown New
2018-03-27 14:40:49 Rafael David Tinoco tags sts sts sts-sponsor
2018-03-27 14:41:03 Rafael David Tinoco bug added subscriber STS Sponsors
2018-03-27 14:41:17 Rafael David Tinoco bug added subscriber Ubuntu Sponsors Team
2018-03-28 21:09:47 Eric Desrochers bug added subscriber Eric Desrochers
2018-03-28 21:11:41 Eric Desrochers bug task added apache2 (Ubuntu Artful)
2018-03-28 21:11:46 Eric Desrochers bug task added apache2 (Ubuntu Bionic)
2018-03-28 21:11:51 Eric Desrochers bug task added apache2 (Ubuntu Xenial)
2018-03-28 21:11:55 Eric Desrochers bug task added apache2 (Ubuntu Trusty)
2018-03-29 14:03:48 Rafael David Tinoco apache2 (Ubuntu Trusty): status New In Progress
2018-03-29 14:03:51 Rafael David Tinoco apache2 (Ubuntu Xenial): status New In Progress
2018-03-29 14:03:53 Rafael David Tinoco apache2 (Ubuntu Artful): status New In Progress
2018-03-29 14:03:55 Rafael David Tinoco apache2 (Ubuntu Trusty): assignee Rafael David Tinoco (inaddy)
2018-03-29 14:03:56 Rafael David Tinoco apache2 (Ubuntu Xenial): assignee Rafael David Tinoco (inaddy)
2018-03-29 14:03:58 Rafael David Tinoco apache2 (Ubuntu Artful): assignee Rafael David Tinoco (inaddy)
2018-03-29 14:04:00 Rafael David Tinoco apache2 (Ubuntu Trusty): importance Undecided Medium
2018-03-29 14:04:01 Rafael David Tinoco apache2 (Ubuntu Xenial): importance Undecided Medium
2018-03-29 14:04:03 Rafael David Tinoco apache2 (Ubuntu Artful): importance Undecided Medium
2018-03-29 15:15:46 Eric Desrochers apache2 (Ubuntu Bionic): status In Progress Fix Committed
2018-03-29 15:41:55 Launchpad Janitor apache2 (Ubuntu Bionic): status Fix Committed Fix Released
2018-03-31 00:44:21 Bug Watch Updater apache2: status New Fix Released
2018-04-05 18:56:21 Brian Murray apache2 (Ubuntu Artful): status In Progress Fix Committed
2018-04-05 18:56:24 Brian Murray bug added subscriber Ubuntu Stable Release Updates Team
2018-04-05 18:56:25 Brian Murray bug added subscriber SRU Verification
2018-04-05 18:56:29 Brian Murray tags sts sts-sponsor sts sts-sponsor verification-needed verification-needed-artful
2018-04-05 19:06:36 Brian Murray apache2 (Ubuntu Xenial): status In Progress Fix Committed
2018-04-05 19:06:42 Brian Murray tags sts sts-sponsor verification-needed verification-needed-artful sts sts-sponsor verification-needed verification-needed-artful verification-needed-xenial
2018-04-05 19:09:01 Brian Murray apache2 (Ubuntu Trusty): status In Progress Fix Committed
2018-04-05 19:09:07 Brian Murray tags sts sts-sponsor verification-needed verification-needed-artful verification-needed-xenial sts sts-sponsor verification-needed verification-needed-artful verification-needed-trusty verification-needed-xenial
2018-04-05 20:38:29 Brian Murray removed subscriber Ubuntu Sponsors Team
2018-04-08 23:48:17 Rafael David Tinoco tags sts sts-sponsor verification-needed verification-needed-artful verification-needed-trusty verification-needed-xenial sts sts-sponsor verification-done
2018-04-11 18:41:44 Dan Streetman tags sts sts-sponsor verification-done sts sts-sponsor verification-done verification-done-artful verification-done-trusty verification-done-xenial
2018-04-19 07:54:39 Launchpad Janitor apache2 (Ubuntu Artful): status Fix Committed Fix Released
2018-04-19 07:54:45 Łukasz Zemczak removed subscriber Ubuntu Stable Release Updates Team
2018-04-19 07:54:56 Launchpad Janitor apache2 (Ubuntu Xenial): status Fix Committed Fix Released
2018-04-19 07:55:59 Launchpad Janitor apache2 (Ubuntu Trusty): status Fix Committed Fix Released
2018-05-09 18:57:40 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345310
2018-05-09 18:58:19 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345311
2018-05-09 18:59:32 Andreas Hasenack merge proposal unlinked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345311
2018-05-09 19:02:17 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345312
2018-05-09 19:22:18 Andreas Hasenack merge proposal unlinked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345312
2018-05-11 19:34:38 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345312
2018-05-15 17:53:45 Andreas Hasenack merge proposal unlinked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345312
2018-05-17 14:04:21 Launchpad Janitor merge proposal linked https://code.launchpad.net/~ahasenack/ubuntu/+source/apache2/+git/apache2/+merge/345312
2019-03-20 13:12:54 Dan Streetman tags sts sts-sponsor verification-done verification-done-artful verification-done-trusty verification-done-xenial sts verification-done verification-done-artful verification-done-trusty verification-done-xenial
2019-03-20 13:13:01 Dan Streetman removed subscriber STS Sponsors