LTE: ISST:leeklp4 no mpath choices for install ubuntu 15.10

Bug #1489379 reported by bugproxy on 2015-08-27
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
multipath-tools (Ubuntu)
High
Mathieu Trudel-Lapierre

Bug Description

== Comment: #0 - Alton L. Pundt <email address hidden> - 2015-08-24 15:12:28 ==
---Problem Description---
no choice for mpath disks.
They all show up as single disks.

Contact Information = A.P. Pundt <email address hidden>

---uname output---
4.1.0-3-generic

Machine Type = 8408-E8E

---boot type---
Network boot

---bootloader---
grub

---Kernel cmdline used to launch install---
linux ubuntu-installer/ppc64el/vmlinux tasks=standard pkgsel\|
 |/language-pack-patterns= pkgsel/install-language-support=false url=http://1\|
 |0.33.11.31/distros/ubuntu/preseed.cfg preseed/url/checksum=edc0fad5d7d5709d\|
 |042c734730e8fb3e disk-detect/multipath/enable=true --- quiet |
 | initrd ubuntu-installer/ppc64el/initrd.gz |
 |

---Bootloader protocol---
http

---Install repository type---
Internet repository

---Install repository Location---
United Kingdon ports.ubuntu.com

---Point of failure---
Other failure during installation (stage 1)

== Comment: #2 - Alton L. Pundt <email address hidden> - 2015-08-24 15:16:58 ==
installer user and password is lte:don2rry

[apundt@kte ~]$ sudo config_dhcpd.py list -H leeklp4
  Host Label : leeklp4
Network Type : ethernet
 MAC Address : 00:90:fa:95:60:16
  IP Address : 10.33.32.110
      Server : 10.33.11.31
    Filename : ubuntu-installer/ppc64el/powerpc-ieee1275/core.elf

[apundt@kte ~]$

== Comment: #3 - Mauricio Faria De Oliveira <email address hidden> - 2015-08-26 12:15:08 ==
Problem here is multipath is not obtaining the paths' identifiers, thus blacklists them all.
I have a good idea of what's happening, from a bug fixed in Debian, probably pending backports to Ubuntu.

I'll check a few things in the LPAR yet.

~ # multipath -v3
...

===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev dev_st
     2:0:7:0 sdm 8:192 -1 undef faulty IBM,2107900 running
     2:0:7:1 sdn 8:208 -1 undef faulty IBM,2107900 running
     2:0:7:2 sdo 8:224 -1 undef faulty IBM,2107900 running
     2:0:7:3 sdp 8:240 -1 undef faulty IBM,2107900 running
     2:0:0:0 sdy 65:128 -1 undef faulty IBM,2107900 running
     2:0:0:1 sdz 65:144 -1 undef faulty IBM,2107900 running
     2:0:0:2 sdaa 65:160 -1 undef faulty IBM,2107900 running
     2:0:0:3 sdab 65:176 -1 undef faulty IBM,2107900 running
     2:0:1:0 sdac 65:192 -1 undef faulty IBM,2107900 running
     2:0:1:1 sdad 65:208 -1 undef faulty IBM,2107900 running
     2:0:1:2 sdae 65:224 -1 undef faulty IBM,2107900 running
     2:0:1:3 sdaf 65:240 -1 undef faulty IBM,2107900 running
     2:0:2:0 sda 8:0 -1 undef faulty IBM,2107900 running
     2:0:2:1 sdb 8:16 -1 undef faulty IBM,2107900 running
     2:0:2:2 sdc 8:32 -1 undef faulty IBM,2107900 running
     2:0:2:3 sdd 8:48 -1 undef faulty IBM,2107900 running
     2:0:3:0 sde 8:64 -1 undef faulty IBM,2107900 running
     2:0:3:0 sde 8:64 -1 undef faulty IBM,2107900 running
     2:0:3:1 sdf 8:80 -1 undef faulty IBM,2107900 running
     2:0:3:2 sdg 8:96 -1 undef faulty IBM,2107900 running
     2:0:3:3 sdh 8:112 -1 undef faulty IBM,2107900 running
     2:0:4:0 sdi 8:128 -1 undef faulty IBM,2107900 running
     2:0:4:1 sdj 8:144 -1 undef faulty IBM,2107900 running
     2:0:4:2 sdk 8:160 -1 undef faulty IBM,2107900 running
     2:0:4:3 sdl 8:176 -1 undef faulty IBM,2107900 running
Aug 26 16:06:41 | sdm: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdn: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdo: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdp: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdy: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdz: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdaa: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdab: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdac: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdad: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdae: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdaf: orphan path, wwid blacklisted
Aug 26 16:06:41 | sda: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdb: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdc: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdd: orphan path, wwid blacklisted
Aug 26 16:06:41 | sde: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdf: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdg: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdh: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdi: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdj: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdk: orphan path, wwid blacklisted
Aug 26 16:06:41 | sdl: orphan path, wwid blacklisted

Also noticed another problem: a broken shared library dependency. (similar story w/ Debian, IIUIC).
I'll open another bug.

Aug 26 16:06:41 | loading /lib/multipath/libchecktur.so checker
Aug 26 16:06:41 | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
Aug 26 16:06:41 | unloading tur checker
Aug 26 16:06:41 | sdm: path checker = (controller setting)

== Comment: #6 - Mauricio Faria De Oliveira <email address hidden> - 2015-08-26 18:43:40 ==
The uuid (identifier) is null because of the shlib error (libgcc_s.so.1 not found)
when loading a path checker.

In libmultipath/discovery.c, pathinfo() will skip assigning an uuid
if get_state() hits 'No checker (is) selected' (which happened; see message).

 get_state() {
  ...
                  condlog(3, "%s: No checker selected", pp->dev);
                  return PATH_UNCHECKED;
  ...
 }

 pathinfo() {
  ...
                  pp->chkrstate = pp->state = get_state(pp, 0);
                  if (pp->state == PATH_UNCHECKED ||
                      pp->state == PATH_WILD)
                          goto blank;
  ...
          get_uid(pp);
  ...
  blank:
  ...
 }

 ... | sdm: get_state
 ... | loading /lib/multipath/libchecktur.so checker
 ... | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
 ... | unloading tur checker
 ...
 ... | sdm: No checker selected
 ... | Discover device <...>/sdm/sdm1
 ... | Discover device <...>/sdm/sdm2
 ... | Discover device <...>/sdm/sdm3
 ... | Discover device <...>/sdn
 ... | sdn: udev property ID_WWN whitelisted

There's no libgcc in the installer.

 ~ # find /lib/ -name '*gcc*'
 ~ #

Tried overriding the path checker, but the setting is taken from the hardcoded value (hwentry),
and even changing the tur checker library for another one which don't depend on libgcc didn't work
 -- it seems the piece dependening on libgcc is required for the load operation, not a library.

 ~ # for lib in /lib/multipath/libcheck*.so; do echo $lib; grep -a libgcc_s.so.1 $lib >/dev/null; echo $?; done
 /lib/multipath/libcheckcciss_tur.so
 1
 /lib/multipath/libcheckdirectio.so
 1
 /lib/multipath/libcheckemc_clariion.so
 1
 /lib/multipath/libcheckhp_sw.so
 1
 /lib/multipath/libcheckrdac.so
 1
 /lib/multipath/libcheckreadsector0.so
 1
 /lib/multipath/libchecktur.so
 0

 ~ # mv /lib/multipath/libchecktur.so /lib/multipath/libchecktur.so.orig
 ~ # ln -s /lib/multipath/libcheckdirectio.so /lib/multipath/libchecktur.so

 ~ # multipath -v3
 ...
 ... | loading /lib/multipath/libchecktur.so checker
 ... | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
 ... | unloading tur checker
 ...

== Comment: #8 - Mauricio Faria De Oliveira <email address hidden> - 2015-08-26 18:57:56 ==
Ubuntu actually does contain the fix for static libgcc from Debian [1],
but it's not working on Ubuntu for some reason.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=779579#50

== Comment: #9 - Mauricio Faria De Oliveira <email address hidden> - 2015-08-26 19:16:42 ==
(In reply to comment #8)
> but it's not working on Ubuntu for some reason.

Or not working for specific files for some reason :)

The -static-libgcc parameter is correctly being carried over in the build, but libchecktur doesn't get it.
And for some reason it's the only libcheck* that uses libgcc.

Checking all libs in multipath-udeb for libgcc:

 ~/mpathudeb$ find lib/ -name '*.so' -print -exec grep -H libgcc '{}' ';'
 lib/multipath/libpriordac.so
 lib/multipath/libpriodatacore.so
 lib/multipath/libcheckrdac.so
 lib/multipath/libcheckemc_clariion.so
 lib/multipath/libpriorandom.so
 lib/multipath/libprioontap.so
 lib/multipath/libprioconst.so
 lib/multipath/libcheckreadsector0.so
 lib/multipath/libcheckhp_sw.so
 lib/multipath/libpriohp_sw.so
 lib/multipath/libpriohds.so
 lib/multipath/libprioweightedpath.so
 lib/multipath/libcheckdirectio.so
 lib/multipath/libchecktur.so
 Binary file lib/multipath/libchecktur.so matches
 lib/multipath/libcheckcciss_tur.so
 lib/multipath/libprioiet.so
 lib/multipath/libprioalua.so
 lib/multipath/libprioemc.so

Checking the build log [1] for -static-libgcc (in there).
Maybe it's also required in the last cc command as well.

 make[2]: Entering directory '/?PKGBUILDDIR?/libmultipath/checkers'
 cc -g -O3 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -static-libgcc -D_FORTIFY_SOURCE=2 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wstrict-prototypes -fPIC -DLIB_STRING=\"lib\" -I.. -c -o libsg.o libsg.c
 ...
 cc -g -O3 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -static-libgcc -D_FORTIFY_SOURCE=2 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wstrict-prototypes -fPIC -DLIB_STRING=\"lib\" -I.. -c -o tur.o tur.c
 cc -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -shared -o libchecktur.so libsg.o tur.o

[1] https://launchpadlibrarian.net/213493168/buildlog_ubuntu-wily-ppc64el.multipath-tools_0.5.0-7ubuntu3_BUILDING.txt.gz

bugproxy (bugproxy) on 2015-08-27
tags: added: architecture-ppc64 bugnameltc-129385 severity-critical targetmilestone-inin1510

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1489379/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

tags: added: bot-comment
affects: ubuntu → multipath-tools (Ubuntu)

------- Comment From <email address hidden> 2015-09-02 22:11 EDT-------
Today's work on this..

This is not so simple to "fix".

libchecktur.so really requires a dynamic libgcc because it uses pthreads (namely pthread_cancel).
it can go without a link-time dependency on libgcc *if* it's not built with -fexceptions.

it will run, but crash when pthread_cancel() is called.
I think this is already happening w/ other pieces of the code (libmultipath, iirc) - it's just luck it didn't crash yet.

Some testing/experiments.

$ cat test.c
#include <pthread.h>
#include <stdlib.h>

void * loop(void *arg) {
}

int main() {
pthread_t thread;
pthread_create(&thread, NULL, &loop, NULL);
pthread_cancel(thread); // this requires libgcc_s.so
return 0;
}

$ gcc -pthread -o test test.c

$ ./test; echo $?
0

$ ldd test
linux-vdso64.so.1 => (0x00003fff81730000)
libpthread.so.0 => /lib/powerpc64le-linux-gnu/libpthread.so.0 (0x00003fff816e0000)
libc.so.6 => /lib/powerpc64le-linux-gnu/libc.so.6 (0x00003fff81500000)
/lib64/ld64.so.2 (0x0000000050130000)

^ notice there's no dependency on libgcc_s at link-time, it's a run-time dlopen() in pthread_cancel_init()

$ find /lib -name libgcc_s.so.1
/lib/powerpc64le-linux-gnu/libgcc_s.so.1

On installer:

~ # ./test
libgcc_s.so.1 must be installed for pthread_cancel to work
Aborted

~ # find /lib -name libgcc_s.so.1
~ #

To be continued.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-03 10:18 EDT-------
Some developer-backed reference for this as well:

(In reply to comment #15)
> libchecktur.so really requires a dynamic libgcc because it uses pthreads
> (namely pthread_cancel).

"""
> Wonder if someone can explain why pthread_cancel_init() dlopens
> libgcc_s.so.1?

Thread cancellation requires unwinding which is implemented by
compiler helper routines.
"""
-- https://sourceware.org/ml/libc-help/2009-10/msg00023.html

bugproxy (bugproxy) wrote :
Download full text (4.1 KiB)

------- Comment From <email address hidden> 2015-09-03 11:05 EDT-------
How to reproduce with QEMU/KVM (2 arguments as "-drive file=test.qcow2,serial=helloworld")

Force the path checker to tur:

~ # cat /etc/multipath.conf
defaults {
user_friendly_names yes
path_checker tur
}

blacklist_exceptions {
property ID_SERIAL
}

Notice the dlopen() error, and paths got no uuids, thus blacklisted:

~ # multipath -v3
...
<...> | sda: udev property ID_SERIAL whitelisted
...
<...> | sda: vendor = QEMU
<...> | sda: product = QEMU HARDDISK
<...> | sda: rev = 2.2.
<...> | sda: h:b:t:l = 0:0:1:0
...
<...> | sda: serial = helloworld
<...> | sda: get_state
<...> | loading /lib/multipath/libchecktur.so checker
<...> | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
<...> | unloading tur checker
<...> | sda: path checker = (config file default)
<...> | sda: checker timeout = 30 ms (internal default)
<...> | sda: No checker selected
<...> | sdb: udev property ID_SERIAL whitelisted
...
<...> | sdb: vendor = QEMU
<...> | sdb: product = QEMU HARDDISK
<...> | sdb: rev = 2.2.
<...> | sdb: h:b:t:l = 0:0:2:0
...
<...> | sdb: serial = helloworld
<...> | sdb: get_state
<...> | loading /lib/multipath/libchecktur.so checker
<...> | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
<...> | unloading tur checker
<...> | sdb: path checker = (config file default)
<...> | sdb: checker timeout = 30 ms (internal default)
<...> | sdb: No checker selected
<...> | loop0: blacklisted, udev property missing
...
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev dev_st
0:0:1:0 sda 8:0 -1 undef faulty QEMU,QEMU HARDDISK running
0:0:2:0 sdb 8:16 -1 undef faulty QEMU,QEMU HARDDISK running
<...> | sda: orphan path, wwid blacklisted
<...> | sdb: orphan path, wwid blacklisted
...
~ #

------- Comment From <email address hidden> 2015-09-03 11:10 EDT-------
And if you manually put libgcc_s.so.1 (from the libgcc1 deb package), it works fine.

~ # ls /lib/powerpc64le-linux-gnu/libgcc_s.so.1
/lib/powerpc64le-linux-gnu/libgcc_s.so.1

~ # multipath -v3
...
<...> | sda: udev property ID_SERIAL whitelisted
...
<...> | sda: serial = helloworld
<...> | sda: get_state
<...> | loading /lib/multipath/libchecktur.so checker
<...> | sda: path checker = tur (config file default)
<...> | sda: checker timeout = 30 ms (internal default)
<...> | sda: state = up
<...> | sda: uid_attribute = ID_SERIAL (internal default)
<...> | sda: uid = 0QEMU_QEMU_HARDDISK_helloworld (udev)
<...> | sda: detect_prio = 1 (config file default)
<...> | sda: prio = const (internal default)
<...> | sda: prio args = (internal default)
<...> | sda: const prio = 1
<...> | sdb: udev property ID_SERIAL whitelisted
...
<...> | sdb: serial = helloworld
<...> | sdb: get_state
<...> | sdb: path checker = tur (config file default)
<...> | sdb: checker timeout = 30 ms (internal default)
<...> | sdb: state = up
<...> | sdb: uid_attribute = ID_SERIAL (internal default)
<...> | sdb: uid = 0QEMU_QEMU_HARDDISK_helloworld (udev)
<...> | sdb: detect_prio = 1 (config file default)
<...> | sdb: prio = co...

Read more...

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-03 11:11 EDT-------
How to put libgcc_s.so.1 (from the udeb package) in the installer environment,
for testing/work-around purposes:

1) Go to https://launchpad.net/ubuntu/wily/ppc64el/libgcc1/
2) Click the latest (top-most) "Version" link (e.g., 1:5.2.1-15ubuntu5)
from the Release "Pocket" (unless you know you should use Proposed)
3) Copy the link address of the ".deb" file in the "Downloadable files" section
(e.g., http://.../libgcc1_5.2.1-15ubuntu5_ppc64el.deb)

4) On the installer, right after network configuration (username configuration stage),
select "Go Back", and "Execute a shell"
5) Run the following commands:
~ # wget http://launchpadlibrarian.net/215539827/libgcc1_5.2.1-15ubuntu5_ppc64el.deb
~ # ar x libgcc1_*.deb
~ # xzcat data.tar.xz | tar x ./lib/powerpc64le-linux-gnu/libgcc_s.so.1 -C /
~ # exit

~ # ls /lib/powerpc64le-linux-gnu/libgcc_s.so.1
/lib/powerpc64le-linux-gnu/libgcc_s.so.1

The result with the previous test binary as well (multipath in previous comment):

~ # ./test
~ #

------- Comment From <email address hidden> 2015-09-03 11:13 EDT-------
Oops

> How to put libgcc_s.so.1 (from the udeb package) in the installer

s/udeb/libgcc1 deb/

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-03 11:26 EDT-------
Fortunately enough, the *only* ELF (binary/shared-library) in the multipath-udeb that requires libgcc_s is... libchecktur.so
(C'mon.. otherwise we wouldn't be doing this :-)

$ dpkg-buildpackage
...
$ find debian/tmp-multipath-udeb/ \
| while read file; do \
file $file | grep -q ELF || continue
objdump -x $file | grep -q 'NEEDED.*libgcc_s' || continue
echo "$file needs libgcc_s"
done
debian/tmp-multipath-udeb/lib/multipath/libchecktur.so needs libgcc_s
$

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-03 11:56 EDT-------
Okay, using a custom libchecktur.so built without -fexceptions..
It works fine in the installer environment, for its purposes.

This should be OK as there's no multipathd (which exercises the path checkers more) running in the installer.
I'll formalize/support this in another comment.

~ # ls /lib/powerpc64le-linux-gnu/libgcc_s.so.1
ls: /lib/powerpc64le-linux-gnu/libgcc_s.so.1: No such file or directory

~ # grep libgcc_s /lib/multipath/libchecktur.so
~ #

~ # grep libgcc_s /lib/multipath/libchecktur.so.orig
libgcc_s.so.1
~ #

~ # multipath -v3
...
<...> | sda: get_state
<...> | loading /lib/multipath/libchecktur.so checker
<...> | sda: path checker = tur (config file default)
...
<...> | sdb: get_state
<...> | sdb: path checker = tur (config file default)
...
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/re
0QEMU_QEMU_HARDDISK_helloworld 0:0:1:0 sda 8:0 1 undef ready QEMU,QEMU HA
0QEMU_QEMU_HARDDISK_helloworld 0:0:2:0 sdb 8:16 1 undef ready QEMU,QEMU HA
...
create: mpatha (0QEMU_QEMU_HARDDISK_helloworld) undef QEMU,QEMU HARDDISK
size=128G features='0' hwhandler='0' wp=undef
|-+- policy='round-robin 0' prio=1 status=undef
| `- 0:0:1:0 sda 8:0 undef ready running
`-+- policy='round-robin 0' prio=1 status=undef
`- 0:0:2:0 sdb 8:16 undef ready running
<...> | tur checker refcount 2
<...> | const prioritizer refcount 2
<...> | tur checker refcount 1
<...> | const prioritizer refcount 1
<...> | unloading const prioritizer
<...> | unloading tur checker
<...> | unloading directio checker

~ # echo $?
0

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-03 12:25 EDT-------
> Okay, using a custom libchecktur.so built without -fexceptions..
> It works fine in the installer environment, for its purposes.

> This should be OK as there's no multipathd (which exercises the path checkers more) running in the installer.
> I'll formalize/support this in another comment.

The only places that pthread_cancel() is called in libchecktur.so (tur.c + libsg.c) are 2 functions in tur.c:
1) libcheck_free()
2) libcheck_check()

And their calls to pthread_cancel() are /not/ exercised when /not/ in daemon mode
(i.e., running as multipath, not multipathd; as in the installer).

It's only in daemon mode that the checker goes into async mode -- via get_state() -> checker_set_async().
- And it's only in async mode that libcheck_check() might call pthread_cancel(),
- And it's only in async mode that libcheck_check() might set 'holders' to a value
for which libcheck_free() might call pthread_cancel() (holders > 1).

file libmultipath/discovery.c:

get_state (struct path * pp, int daemon)
{
...
condlog(3, "%s: get_state", pp->dev);

if (!checker_selected(c)) {
...
select_checker(pp);
if (!checker_selected(c)) {
condlog(3, "%s: No checker selected", pp->dev);
return PATH_UNCHECKED;
}
...
if (daemon)
checker_set_async(c);

file libmultipath/checkers.c:

void checker_set_async (struct checker * c)
{
if (!c)
return;
c->sync = 0;
}

file libmultipath/checkers/tur.c:

extern int
libcheck_check (struct checker * c)
{
...
if (c->sync)
return tur_check(c->fd, c->timeout, c->message);

/*
* Async mode
*/
...
if (ct->running) {
...
if (tur_check_async_timeout(c)) {
condlog(3, "%d:%d: tur checker timeout",
TUR_DEVT(ct));
pthread_cancel(ct->thread);
...
/* Start new TUR checker */
...
ct->holders++;

int libcheck_init (struct checker * c)
{
...
ct->holders = 1;
...

void libcheck_free (struct checker * c)
...
ct->holders--;
...
if (holders)
pthread_cancel(thread);
...

------- Comment From <email address hidden> 2015-09-03 12:28 EDT-------
Given the supporting evidence provided, I'll work on a patch to build multipath-udeb without -fexceptions.

(In reply to comment #22)
> Fortunately enough, the *only* ELF (binary/shared-library) in the
> multipath-udeb that requires libgcc_s is... libchecktur.so

(In reply to comment #23)
> Okay, using a custom libchecktur.so built without -fexceptions..
> It works fine in the installer environment, for its purposes.
>
> This should be OK as there's no multipathd (which exercises the path
> checkers more) running in the installer.

(In reply to comment #24)
> The only places that pthread_cancel() is called in libchecktur.so (tur.c +
> libsg.c) are 2 functions in tur.c:
...
> And their calls to pthread_cancel() are /not/ exercised when /not/ in daemon
> mode
> (i.e., running as multipath, not multipathd; as in the installer).

Download full text (3.6 KiB)

I'm attaching a patch that solves the issue; verification steps below.
It disables -fexceptions when building multipath-udeb.

(I had in mind something more elegant, like only adding -fno-exceptions from debian/rules into DEB_CFLAGS_MAINT_APPEND as done w/ -static-libgcc, but that way it doesn't override the post-added -fexceptions in Makefile.inc).. and found no other variable that would override it w/ less changes. So it's adding a patch to Makefile.inc and it does the job.)

Some comments with more details/explanation of the issue were not mirrored yet; should be on their way.

Build verification:
------------------

Checking the build log, the 2 tur.o objects are built correctly:
1) multipath-udeb: with -static-libgcc and without -fexceptions
2) multipath deb: without -static-libgcc and with -fexceptions (default)

 $ dpkg-buildpackage 2>&1 | tee ../dpkg-buildpackage.log

 $ grep -- '-o tur.o' ../dpkg-buildpackage.log
 cc -g -O3 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -static-libgcc -D_FORTIFY_SOURCE=2 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wstrict-prototypes -fPIC -DLIB_STRING=\"lib\" -I.. -c -o tur.o tur.c
 cc -g -O3 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wstrict-prototypes -fPIC -DLIB_STRING=\"lib\" -I.. -c -o tur.o tur.c

Checking the build output, the ELF objects (don't) dyn-link to libgcc_s as appropriate:
1) multipath-udeb: ELF objects don't dyn-link with libgcc_s
2) multipath deb: ELF objects dyn-link with libgcc_s

 $ find debian/tmp-multipath-udeb/ | while read file; do file $file | grep -q ELF && objdump -x $file | grep -q 'NEEDED.*libgcc_s' && echo $file needs libgcc_s; done
 $

 $ find debian/tmp/ | while read file; do file $file | grep -q ELF && objdump -x $file | grep -q 'NEEDED.*libgcc_s' && echo $file needs libgcc_s; done
 debian/tmp/lib/libmultipath.so.0 needs libgcc_s
 debian/tmp/lib/multipath/libchecktur.so needs libgcc_s
 debian/tmp/sbin/multipathd needs libgcc_s
 $

Installer verification:
-----------------------

Configure multipath.conf for using tur with QEMU/KVM SCSI disks:

 ~ # cat <<EOF >/etc/multipath.conf <<EOF
 defaults {
   user_friendly_names yes
   path_checker tur
 }
 blacklist_exceptions {
   property ID_SERIAL
 }
 EOF

Test (Not OK):

 ~ # multipath -v3
 ...
 <...> | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
 ...
 <...> | A dynamic linking error occurred: (libgcc_s.so.1: cannot open shared object file: No such file or directory)
 uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev dev_st
      0:0:1:0 sda 8:0 -1 undef faulty QEMU,QEMU HARDDISK running
      0:0:2:0 sdb 8:16 -1 undef faulty QEMU,QEMU HARDDISK running
 ...

Install the new udeb:

 ~ # udpkg -i multipath-udeb_0.5.0-7ubuntu3fexceptions1_ppc64el.udeb

Re-test (now OK):

 ~ # multipath -v3
 ...
 ===== paths list =====
 uuid hcil dev dev_t pri dm_st chk_st vend/prod/re
 0QEMU_QEMU_HARDDISK_hellowo...

Read more...

The attachment "Disable -fexceptions when building multipath-udeb" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch

Assigning to me and unsubscribing sponsors, I'll just do the review and upload the patch as necessary.

Changed in multipath-tools (Ubuntu):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Mathieu Trudel-Lapierre (mathieu-tl)
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package multipath-tools - 0.5.0-7ubuntu4

---------------
multipath-tools (0.5.0-7ubuntu4) wily; urgency=medium

  * Disable -fexceptions on multipath-udeb (LP: #1489379): the flag causes
    libchecktur.so to link with libgcc_s.so.1 (even with -static-libgcc),
    which is not available in the installer environment.
    - debian/patches/disable-fexceptions-udeb.patch: conditionally disable
      -fexceptions with CFLAGS_DISABLE_FEXCEPTIONS.
    - debian/rules: set CFLAGS_DISABLE_FEXCEPTIONS to build multipath-udeb.

 -- Mauricio Faria de Oliveira <email address hidden> Thu, 03 Sep 2015 07:29:59 -0400

Changed in multipath-tools (Ubuntu):
status: Triaged → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-16 01:07 EDT-------
I can see the multipath choices on the installation menu and able to install the rootvg on one of the multipath disk using SAN storages. System boots up normal after all.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2015-09-18 21:27 EDT-------
*** Bug 129488 has been marked as a duplicate of this bug. ***

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers