| 2017-04-28 11:56:35 |
Rafael David Tinoco |
bug |
|
|
added bug |
| 2017-04-28 12:19:24 |
Rafael David Tinoco |
bug |
|
|
added subscriber Szilard Cserey |
| 2017-04-28 12:19:32 |
Rafael David Tinoco |
tags |
|
sts |
|
| 2017-04-28 12:19:38 |
Rafael David Tinoco |
multipath-tools (Ubuntu): status |
New |
In Progress |
|
| 2017-04-28 12:19:44 |
Rafael David Tinoco |
multipath-tools (Ubuntu): importance |
Undecided |
Medium |
|
| 2017-04-28 12:19:47 |
Rafael David Tinoco |
multipath-tools (Ubuntu): assignee |
|
Rafael David Tinoco (inaddy) |
|
| 2017-04-28 12:19:56 |
Rafael David Tinoco |
multipath-tools (Ubuntu): milestone |
|
ubuntu-14.04.5 |
|
| 2017-04-28 12:20:04 |
Rafael David Tinoco |
nominated for series |
|
Ubuntu Trusty |
|
| 2017-05-01 17:42:24 |
Rafael David Tinoco |
attachment added |
|
trusty_multipath-tools_0.4.9-3ubuntu7.16.debdiff https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1687004/+attachment/4870427/+files/trusty_multipath-tools_0.4.9-3ubuntu7.16.debdiff |
|
| 2017-05-02 06:39:39 |
Dominique Poulain |
bug |
|
|
added subscriber Dominique Poulain |
| 2017-06-20 12:34:17 |
Rafael David Tinoco |
tags |
sts |
sts sts-sponsor |
|
| 2017-06-20 12:34:27 |
Rafael David Tinoco |
tags |
sts sts-sponsor |
sts |
|
| 2017-06-20 13:02:38 |
Rafael David Tinoco |
description |
It was brought to my attention that:
multipath-tools: 0.4.9-3ubuntu7.15
Faced a crash and generated a dump.
## multipath (trusty) crashed and its dump shows:
(gdb) bt full
#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:66
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
type = 0
#1 0x00007f48700b606e in flush_logqueue () at log_pthread.c:39
empty = 0
#2 0x00007f48700b611b in log_thread (et=0x0) at log_pthread.c:57
No locals.
#3 0x00007f4870964184 in start_thread (arg=0x7f4870d8b700) at pthread_create.c:312
__res = <optimized out>
pd = 0x7f4870d8b700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139949107623680, -3200163692152804016, 0, 0, 139949107624384, 139949107623680, 3244383534590274896,
3244383107817352528}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#4 0x00007f486fdb537d in __ecvt_r (value=9.532824124368238e-130, ndigit=0, decpt=0x0, sign=0x0, buf=0x7f4870d8b9c0 "\220R\267pH\177", len=139949107623680)
at efgcvt_r.c:218
d = 0
f = 3.2378592100206092e-319
exponent = 1893250816
#5 0x0000000000000000 in ?? ()
No symbol table info available. |
[Impact]
* mutexes are being allocated from memory with no checks and, if allocation fails, it causes these kind of segfaults later in the code execution.
expect multipath daemon to crash and not run any checkers on path groups.
* not checking path groups, in an event of failure, the mpath won't change path prios.
* openstack relies on flushing device maps frequently when using iscsi.
[Test Case]
* i'm fixing this based on a dump analysis and not on reproduction.
* if you disallow memory overcommit - facilitating memory exhaustion - you would be able to reproduce that by stressing multipathd with paths being flushed, but that is theory only.
[Regression Potential]
* the patch is changing the locking mechanism for log thread, based on upstream commit.
* major change is to use the mutexes from stack instead of allocating from heap.
* multipath log thread could not work as designed.
* tested by reported and reported to be good.
[Other Info]
It was brought to my attention that:
multipath-tools: 0.4.9-3ubuntu7.15
Faced a crash and generated a dump.
## multipath (trusty) crashed and its dump shows:
(gdb) bt full
#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:66
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
type = 0
#1 0x00007f48700b606e in flush_logqueue () at log_pthread.c:39
empty = 0
#2 0x00007f48700b611b in log_thread (et=0x0) at log_pthread.c:57
No locals.
#3 0x00007f4870964184 in start_thread (arg=0x7f4870d8b700) at pthread_create.c:312
__res = <optimized out>
pd = 0x7f4870d8b700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139949107623680, -3200163692152804016, 0, 0, 139949107624384, 139949107623680, 3244383534590274896,
3244383107817352528}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#4 0x00007f486fdb537d in __ecvt_r (value=9.532824124368238e-130, ndigit=0, decpt=0x0, sign=0x0, buf=0x7f4870d8b9c0 "\220R\267pH\177", len=139949107623680)
at efgcvt_r.c:218
d = 0
f = 3.2378592100206092e-319
exponent = 1893250816
#5 0x0000000000000000 in ?? ()
No symbol table info available. |
|
| 2017-06-20 13:12:20 |
Rafael David Tinoco |
attachment removed |
trusty_multipath-tools_0.4.9-3ubuntu7.16.debdiff https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1687004/+attachment/4870427/+files/trusty_multipath-tools_0.4.9-3ubuntu7.16.debdiff |
|
|
| 2017-06-20 13:30:08 |
Eric Desrochers |
bug task added |
|
multipath-tools (Ubuntu Trusty) |
|
| 2017-06-20 14:04:37 |
Eric Desrochers |
multipath-tools (Ubuntu Trusty): assignee |
|
Rafael David Tinoco (inaddy) |
|
| 2017-06-20 14:04:41 |
Eric Desrochers |
multipath-tools (Ubuntu Trusty): status |
New |
In Progress |
|
| 2017-06-20 14:04:44 |
Eric Desrochers |
multipath-tools (Ubuntu Trusty): importance |
Undecided |
Medium |
|
| 2017-06-20 14:04:47 |
Eric Desrochers |
multipath-tools (Ubuntu): status |
In Progress |
Fix Released |
|
| 2017-06-20 14:04:50 |
Eric Desrochers |
multipath-tools (Ubuntu): assignee |
Rafael David Tinoco (inaddy) |
|
|
| 2017-06-20 14:14:31 |
Eric Desrochers |
tags |
sts |
sts sts-sru-needed |
|
| 2017-06-20 20:25:58 |
Eric Desrochers |
description |
[Impact]
* mutexes are being allocated from memory with no checks and, if allocation fails, it causes these kind of segfaults later in the code execution.
expect multipath daemon to crash and not run any checkers on path groups.
* not checking path groups, in an event of failure, the mpath won't change path prios.
* openstack relies on flushing device maps frequently when using iscsi.
[Test Case]
* i'm fixing this based on a dump analysis and not on reproduction.
* if you disallow memory overcommit - facilitating memory exhaustion - you would be able to reproduce that by stressing multipathd with paths being flushed, but that is theory only.
[Regression Potential]
* the patch is changing the locking mechanism for log thread, based on upstream commit.
* major change is to use the mutexes from stack instead of allocating from heap.
* multipath log thread could not work as designed.
* tested by reported and reported to be good.
[Other Info]
It was brought to my attention that:
multipath-tools: 0.4.9-3ubuntu7.15
Faced a crash and generated a dump.
## multipath (trusty) crashed and its dump shows:
(gdb) bt full
#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:66
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
type = 0
#1 0x00007f48700b606e in flush_logqueue () at log_pthread.c:39
empty = 0
#2 0x00007f48700b611b in log_thread (et=0x0) at log_pthread.c:57
No locals.
#3 0x00007f4870964184 in start_thread (arg=0x7f4870d8b700) at pthread_create.c:312
__res = <optimized out>
pd = 0x7f4870d8b700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139949107623680, -3200163692152804016, 0, 0, 139949107624384, 139949107623680, 3244383534590274896,
3244383107817352528}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#4 0x00007f486fdb537d in __ecvt_r (value=9.532824124368238e-130, ndigit=0, decpt=0x0, sign=0x0, buf=0x7f4870d8b9c0 "\220R\267pH\177", len=139949107623680)
at efgcvt_r.c:218
d = 0
f = 3.2378592100206092e-319
exponent = 1893250816
#5 0x0000000000000000 in ?? ()
No symbol table info available. |
[Impact]
* mutexes are being allocated from memory with no checks and, if allocation fails, it causes these kind of segfaults later in the code execution.
expect multipath daemon to crash and not run any checkers on path groups.
* not checking path groups, in an event of failure, the mpath won't change path prios.
* openstack relies on flushing device maps frequently when using iscsi.
[Test Case]
* i'm fixing this based on a dump analysis and not on reproduction.
* if you disallow memory overcommit - facilitating memory exhaustion - you would be able to reproduce that by stressing multipathd with paths being flushed, but that is theory only.
[Regression Potential]
* the patch is changing the locking mechanism for log thread, based on upstream commit.
* major change is to use the mutexes from stack instead of allocating from heap.
* multipath log thread could not work as designed.
* tested by reported and reported to be good.
* What releases are affected ?
The following releases already got the fix
- Xenial/Yakkety/Zesty/Artful
Note that Debian also has the fix.
Meaning that ONLY Trusty is affected by this bug.
* This SRU contained fixes for 2 LP bugs:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1687004
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1695789
[Other Info]
It was brought to my attention that:
multipath-tools: 0.4.9-3ubuntu7.15
Faced a crash and generated a dump.
## multipath (trusty) crashed and its dump shows:
(gdb) bt full
#0 __GI___pthread_mutex_lock (mutex=0x0) at ../nptl/pthread_mutex_lock.c:66
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
type = 0
#1 0x00007f48700b606e in flush_logqueue () at log_pthread.c:39
empty = 0
#2 0x00007f48700b611b in log_thread (et=0x0) at log_pthread.c:57
No locals.
#3 0x00007f4870964184 in start_thread (arg=0x7f4870d8b700) at pthread_create.c:312
__res = <optimized out>
pd = 0x7f4870d8b700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139949107623680, -3200163692152804016, 0, 0, 139949107624384, 139949107623680, 3244383534590274896,
3244383107817352528}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#4 0x00007f486fdb537d in __ecvt_r (value=9.532824124368238e-130, ndigit=0, decpt=0x0, sign=0x0, buf=0x7f4870d8b9c0 "\220R\267pH\177", len=139949107623680)
at efgcvt_r.c:218
d = 0
f = 3.2378592100206092e-319
exponent = 1893250816
#5 0x0000000000000000 in ?? ()
No symbol table info available. |
|
| 2017-06-20 20:26:56 |
Eric Desrochers |
bug |
|
|
added subscriber Eric Desrochers |
| 2017-06-22 16:35:03 |
Łukasz Zemczak |
multipath-tools (Ubuntu Trusty): status |
In Progress |
Fix Committed |
|
| 2017-06-22 16:35:04 |
Łukasz Zemczak |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2017-06-22 16:35:06 |
Łukasz Zemczak |
bug |
|
|
added subscriber SRU Verification |
| 2017-06-22 16:35:13 |
Łukasz Zemczak |
tags |
sts sts-sru-needed |
sts sts-sru-needed verification-needed |
|
| 2017-07-04 15:33:14 |
Eric Desrochers |
tags |
sts sts-sru-needed verification-needed |
sts sts-sru-needed verification-done-trusty |
|
| 2017-07-04 16:00:27 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2017-07-04 16:00:35 |
Launchpad Janitor |
multipath-tools (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
| 2020-05-21 05:48:28 |
Rafael David Tinoco |
multipath-tools (Ubuntu Trusty): assignee |
Rafael David Tinoco (rafaeldtinoco) |
|
|
| 2020-05-21 05:48:37 |
Rafael David Tinoco |
multipath-tools (Ubuntu): milestone |
ubuntu-14.04.5 |
|
|
