[MIR] google-perftools, libunwind

[James Page]

[Overall Rationale]
Required to support full functional/performant ceph package.

Dependency on google-perftools was dropped during 12.04 cycle to support late MIR; it would be good to re-enable these features for 12.10.

>> google-perftools - http://code.google.com/p/gperftools <<

[Availability]
In universe since hardy (maybe before)

[Rationale]
Provides tcmalloc - a high performance implementation of malloc for multi-threaded environments. This is important from a performance perspective for ceph.

http://gperftools.googlecode.com/svn/trunk/doc/tcmalloc.html

[Security]
No CVE's found.

[Quality assurance]
Package has test suite but currently not enabled: BLOCKED
Blocked by http://code.google.com/p/gperftools/issues/detail?id=428&q=heap-checker_unittest - issues with libc6 2.15 compatibility.

[Dependencies]
Other than libunwind (on this MIR):
Runtime dependency on gv for google-perftools - this can probably be dropped to Suggests (currently recommends).

[Standards compliance]
Package build is CDBS rather than debhelper but looks OK.
Two lintian warnings and a few informational messages - nothing significant.

[Maintenance]
No outstanding bugs in Debian and looks to be well maintained.
Current issue in Lucid - its not installable - but not relevant for 12.10 MIR.
Upstream primary maintainer has just changed - Feb 2012

[Background information]
Quote "Perftools is a collection of a high-performance multi-threaded malloc() implementation, plus some pretty nifty performance analysis tools."
Renamed upstream to gperftools.

>> libunwind - http://www.nongnu.org/libunwind/ <<

[Availability]
In universe since hardy

[Rationale]
Dependency for google-perftools on amd64 only; required for ceph.

[Security]
No CVE's found.

[Quality assurance]
Test suite is currently not enabled and has test failures - this needs to be reported upstream: TODO

[Dependencies]
All in main.

[Standards compliance]
Pre-debhelper 7 style, but relatively simple - looks OK.

[Maintenance]
Maintained in Debian (no outstanding bugs).

[Background information]
Only used in amd64 build - need to investigate why.

[Michael Terry (mterry)]

I thought I had already commented on this one, but I don't see it.

What is the story with the libunwind-setjmp0's diversions of /usr/lib/libunwind-setjmp.so.0.0.0 and /usr/lib/libunwind-setjmp.so.0? Do any other packages provide those files?

[James Page (james-page)]

I think that is probably historic:

libunwind (0.98.5-4) experimental; urgency=low

  * Install the unwind library in the libunwind7 package, not libgcc1.

 -- Matthias Klose <email address hidden> Tue, 3 May 2005 14:50:20 +0000

So is probably no longer required - but I would defer to Matthias who probably understands this better that I do.

[Michael Terry (mterry)]

Enabling the tests for perftools involves a few things:

1) Add "DEB_MAKE_CHECK_TARGET := check" to debian/rules
2) Backport a patch from upstream that fixes tests on i386: http://code.google.com/p/gperftools/source/detail?r=150

But even then, you run into a deadlock: http://code.google.com/p/gperftools/issues/detail?id=423

That's when I stopped working on it. This should probably be fixed/worked-around and tests enabled.

[Michael Terry (mterry)]

And for libunwind, adding "$(MAKE) check" after the "$(MAKE)" call will enable tests. But they have 6 failures, including some segfaults. That should be investigated as well.

[James Page (james-page)]

Micahel

I spent some time digging into the libunwind test failures; from the large amount of discussion about failing tests on the mailing list it appears that there are an expected set of failures which vary depending on architecture.

On x86_64 (5 failures); I was able to cherry pick two fixes (one an actual bug and the other a work-around in the test for calls to calloc ); 2 tests are expected to fail (unsupported features on this arch) and the last is just broken (I see attempts to resolve in trunk).

I was not able to compile some of the tests on arm and there is alot of discussion about hanging and failing tests on the ML for this arch.

Looking at i386 and powerpc now.

[James Page (james-page)]

After cherry picking one more patch for ARM this branch:

lp:~james-page/ubuntu/quantal/libunwind/tests

Produces the results in the attached document.

[James Page (james-page)]

I've reported the libunwind test failures upstream; however my current feel is that it won't be possible to gate all architectures using the test suite as it stands today.

[James Page (james-page)]

I'm not convinced that the deadlock issue in google-perftools is not caused by the combination of libunwind/glibc that is present in 12.10.

The INSTALL, bug tracker and ML make reference to this being a pain point on x86_64 - but I see the same issues on other archs.

[James Page (james-page)]

This one line patch for eglibc (included in 2.16) may help with the google-perftools tests:

  http://permalink.gmane.org/gmane.comp.lib.glibc.alpha/17093

Testing now.

[James Page (james-page)]

The patch to eglibc fixes the issues with hanging/deadlocked test - two of which fail for other reasons.

I'll investigate whether quantal will ship eglibc 2.16 or whether we need to pickup this patch for 2.15

[James Page (james-page)]

Now hitting this bug in google-perftools - http://code.google.com/p/gperftools/issues/detail?id=437

[James Page (james-page)]

Reported glibc bug 1028038

[James Page (james-page)]

I now have a fully functional test suite for google-perftools; proposed patches submitted back upstream.

Once the glibc fix has been uploaded I will upload this package with the test suite enabled.

[James Page (james-page)]

libunwind upstream expects a certain number of know failures for each of the primary archs it supports.

Some of these are documented in the README provided by upstream

x86_64 has the least test failures; I've been able to either cherry pick fixes or provider workarounds in most of the tests. Two are testing unsupported features - so can be disabled.

Upstream's recommendation was to enable the amd64 test suite.

[James Page (james-page)]

New version of libunwind uploaded with test suite enabled for amd64 only:

libunwind (1.0.1-2ubuntu2) quantal; urgency=low

  * Enable test suite for amd64 architecture:
    - d/rules: Execute test suite when building for amd64 only.
    - d/patches/20122307_fix_setjmp.patch: Cherry picked fix from upstream
      VCS to fixup setjmp test failure.
    - d/patches/20123007_disable_known_test_failures.patch: Disable *-dyn1
      tests as they use unsupported features on amd64.
    - d/patches/20123007_workaround_ltest-nocalloc.patch: Make dummy call to
      do_backtrace to ensure that malloc is not called post thread init.
    - d/patches/20123007_workaround_test-ptrace.patch: Walk default test
      arguments to ensure child process has visibility.

Date: Mon, 30 Jul 2012 14:43:28 +0100
Changed-By: James Page <email address hidden>
Maintainer: Ubuntu Developers <email address hidden>
https://launchpad.net/ubuntu/quantal/+source/libunwind/1.0.1-2ubuntu2

[Michael Terry (mterry)]

libunwind is fine with tests enabled. Not all platforms, but upstream makes it difficult to do so. So we at least enable the one platform we need it for, and that one platform's tests will have to act as our canary in a coal mine.

[Launchpad Janitor (janitor)]

This bug was fixed in the package google-perftools - 2.0-3ubuntu1

---------------
google-perftools (2.0-3ubuntu1) quantal; urgency=low

  * Enable test suite execution during package build process (LP: #1017972):
    - d/rules: Execute check target after build, restrict test execution to
      amd64 and i386 architectures only.
    - d/p/20120725_heap-check-after-destructors.patch: Always run heap
      check after destructors as atexit ordering is no longer deterministic.
    - d/p/20120725_ptrace-permissions.patch: Ensure child process can
      ptrace parent pid when heap checking.
    - d/p/20123107_fix_i386_frames.patch: Cherry picked fix from upstream
      VCS to ensure stacks can be traced on i386 architecture.
 -- James Page <email address hidden> Mon, 13 Aug 2012 13:45:29 +0100

[James Page (james-page)]

Michael

My last upload enabled test suite execution for amd64 and i386 architectures; powerpc has some test failures which I think are related to the maturity of support for powerpc in this package upstream (4/44 tests fail). I will report this upstream and file an associated LP bug.

Note that for ceph google-perftools is only used for amd64/i386 archs so hopefully enabling the test suites for these two archs will be sufficient for its inclusion in Ubuntu main.

Thanks

[Jamie Strandboge (jdstrand)]

Override component to main
google-perftools 2.0-3ubuntu1 in quantal: universe/misc -> main
google-perftools 2.0-3ubuntu1 in quantal amd64: universe/devel/optional -> main
google-perftools 2.0-3ubuntu1 in quantal armel: universe/devel/optional -> main
google-perftools 2.0-3ubuntu1 in quantal armhf: universe/devel/optional -> main
google-perftools 2.0-3ubuntu1 in quantal i386: universe/devel/optional -> main
google-perftools 2.0-3ubuntu1 in quantal powerpc: universe/devel/optional -> main
libgoogle-perftools-dev 2.0-3ubuntu1 in quantal amd64: universe/libdevel/optional -> main
libgoogle-perftools-dev 2.0-3ubuntu1 in quantal i386: universe/libdevel/optional -> main
libgoogle-perftools-dev 2.0-3ubuntu1 in quantal powerpc: universe/libdevel/optional -> main
libgoogle-perftools4 2.0-3ubuntu1 in quantal amd64: universe/libs/optional -> main
libgoogle-perftools4 2.0-3ubuntu1 in quantal i386: universe/libs/optional -> main
libgoogle-perftools4 2.0-3ubuntu1 in quantal powerpc: universe/libs/optional -> main
libgoogle-perftools4-dbg 2.0-3ubuntu1 in quantal amd64: universe/debug/extra -> main
libgoogle-perftools4-dbg 2.0-3ubuntu1 in quantal i386: universe/debug/extra -> main
libgoogle-perftools4-dbg 2.0-3ubuntu1 in quantal powerpc: universe/debug/extra -> main
libtcmalloc-minimal4 2.0-3ubuntu1 in quantal amd64: universe/libs/optional -> main
libtcmalloc-minimal4 2.0-3ubuntu1 in quantal i386: universe/libs/optional -> main
libtcmalloc-minimal4 2.0-3ubuntu1 in quantal powerpc: universe/libs/optional -> main
libtcmalloc-minimal4-dbg 2.0-3ubuntu1 in quantal amd64: universe/debug/extra -> main
libtcmalloc-minimal4-dbg 2.0-3ubuntu1 in quantal i386: universe/debug/extra -> main
libtcmalloc-minimal4-dbg 2.0-3ubuntu1 in quantal powerpc: universe/debug/extra -> main
Override [y|N]? y

[Jamie Strandboge (jdstrand)]

Override component to main
libunwind 1.0.1-2ubuntu2 in quantal: universe/libs -> main
libunwind-setjmp0 1.0.1-2ubuntu2 in quantal amd64: universe/libs/optional -> main
libunwind-setjmp0 1.0.1-2ubuntu2 in quantal armel: universe/libs/optional -> main
libunwind-setjmp0 1.0.1-2ubuntu2 in quantal armhf: universe/libs/optional -> main
libunwind-setjmp0 1.0.1-2ubuntu2 in quantal i386: universe/libs/optional -> main
libunwind-setjmp0 1.0.1-2ubuntu2 in quantal powerpc: universe/libs/optional -> main
libunwind-setjmp0-dev 1.0.1-2ubuntu2 in quantal amd64: universe/libdevel/optional -> main
libunwind-setjmp0-dev 1.0.1-2ubuntu2 in quantal armel: universe/libdevel/optional -> main
libunwind-setjmp0-dev 1.0.1-2ubuntu2 in quantal armhf: universe/libdevel/optional -> main
libunwind-setjmp0-dev 1.0.1-2ubuntu2 in quantal i386: universe/libdevel/optional -> main
libunwind-setjmp0-dev 1.0.1-2ubuntu2 in quantal powerpc: universe/libdevel/optional -> main
libunwind8 1.0.1-2ubuntu2 in quantal amd64: universe/libs/optional -> main
libunwind8 1.0.1-2ubuntu2 in quantal armel: universe/libs/optional -> main
libunwind8 1.0.1-2ubuntu2 in quantal armhf: universe/libs/optional -> main
libunwind8 1.0.1-2ubuntu2 in quantal i386: universe/libs/optional -> main
libunwind8 1.0.1-2ubuntu2 in quantal powerpc: universe/libs/optional -> main
libunwind8-dev 1.0.1-2ubuntu2 in quantal amd64: universe/libdevel/optional -> main
libunwind8-dev 1.0.1-2ubuntu2 in quantal armel: universe/libdevel/optional -> main
libunwind8-dev 1.0.1-2ubuntu2 in quantal armhf: universe/libdevel/optional -> main
libunwind8-dev 1.0.1-2ubuntu2 in quantal i386: universe/libdevel/optional -> main
libunwind8-dev 1.0.1-2ubuntu2 in quantal powerpc: universe/libdevel/optional -> main
Override [y|N]? y