FFe: Sync libpfm4 4.7.0+git30-gd422ba2-2 (main) from Debian unstable (main)

[Graham Inggs]

Please sync libpfm4 4.7.0+git30-gd422ba2-2 (main) from Debian unstable (main)

Explanation of FeatureFreeze exception:
Request from Debian maintainer, enables support for latest Intel CPUs, including Knights Landing.

Changelog entries since current yakkety version 4.7.0+git11-gbfb9baf-1:

libpfm4 (4.7.0+git30-gd422ba2-2) unstable; urgency=medium

  * PIE.patch: New, -fPIC needs to override -fPIE.
  * Build with PIE enabled.

 -- Andreas Beckmann <email address hidden> Fri, 07 Oct 2016 15:35:12 +0200

libpfm4 (4.7.0+git30-gd422ba2-1) unstable; urgency=medium

  * New upstream GIT snapshot.
  * Overhaul buildflags usage and enable more hardening.
  * Fix i386 (cross-)build with x86_64 uname.
  * typos.patch: Fix some more typos.

 -- Andreas Beckmann <email address hidden> Mon, 19 Sep 2016 02:21:25 +0200

Also, please sync papi 5.5.0-2 (universe) from Debian unstable (main)

Changelog entries since current yakkety version 5.4.3-3:

papi (5.5.0-2) unstable; urgency=medium

  * Fix upgrade from jessie with Breaks+Replaces: libpapi5.3.
  * Fix build on hppa, thanks to Helge Deller. (Closes: #838321)

 -- Andreas Beckmann <email address hidden> Fri, 23 Sep 2016 12:14:28 +0200

papi (5.5.0-1) unstable; urgency=medium

  * New upstream release.
  * Refresh patches.
  * fix-typos.patch: Fix some more typos.
  * Bump libpfm4-dev B-D to >= 4.7.0+git30.
  * Restore reproducibility.
  * Upload to unstable.

 -- Andreas Beckmann <email address hidden> Mon, 19 Sep 2016 09:40:23 +0200

papi (5.4.3-4) experimental; urgency=medium

  * Upstream has clarified what composes the library API.
  * Soversion change: 5.4.3 -> 5. Rename package libpapi5.4.3 to libpapi5.
  * Drop support for changing the Soversion frequently.
  * Overhaul buildflags usage and enable more hardening.
  * Upload to experimental.

 -- Andreas Beckmann <email address hidden> Sun, 29 May 2016 10:39:59 +0200

All reverse-dependencies built in my PPA, except eztrace-contrib on amd64, due to LP: #1629274.

[Steve Langasek (vorlon)]

The new libpfm4 version seems to include changes besides just adding new code for enabling the new CPUs, and it's non-trivial to see from source review what impact this may have on existing uses. Could we have some analysis of these changes, and their potential risk to the release?

There's also this packaging change:

+# see FEATURE AREAS in dpkg-buildflags(1)
+export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie

This explicitly disables PIE, which is enabled by default on amd64 and s390x in yakkety. Why should we want to disable PIE? There's no explanation in the debian changelog.

[Steve Langasek (vorlon)]

The papi FFe looks straightforward, granted.

[Andreas Beckmann (anbe)]

On 2016-10-03 23:10, Steve Langasek wrote:
> The new libpfm4 version seems to include changes besides just adding new
> code for enabling the new CPUs, and it's non-trivial to see from source
> review what impact this may have on existing uses. Could we have some
> analysis of these changes, and their potential risk to the release?
>
> There's also this packaging change:
>
> +# see FEATURE AREAS in dpkg-buildflags(1)
> +export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
>
> This explicitly disables PIE, which is enabled by default on amd64 and
> s390x in yakkety. Why should we want to disable PIE? There's no
> explanation in the debian changelog.

PIE wasn't enabled in Debian before that change and IIRC I had some
build problems on i386 or amd64 or ??? and therefore explicitly didn't
turn on PIE. I'll have a look at this again within the next weeks.

Andreas

[Graham Inggs (ginggs)]

For reference, these are the commits between 4.7.0+git11-gbfb9baf-1 and 4.7.0+git30-gd422ba2-1.
(Newest at the top)

[d422ba2] fix Intel Goldmont offcore_response average latency support
[ef73810] Add support for Intel Knights Landing core PMU
[359a11a] allow . as a delimiter for event strings
[408701e] fix encodings of L2_RQSTS:PF_MISS and PF_HIT for HSW/BDW
[7ac65a6] add BR_INST_RETIRED:ALL_TAKEN_BRANCHES to Intel Silvermont event table
[6e764d5] add BR_INST_RETIRED:ALL_TAKEN_BRANCHES to Intel Goldmont event table
[25117cf] add SQ_MISC:SPLIT_LOCK to Intel IvyBridge event table
[073e4db] add SQ_MISC:SPLIT_LOCK to Intel Haswell event table
[98a2c64] update Intel Skylake event table
[bdf0395] Add SQ_MISC:SPLIT_LOCK to Intel Broadwell event table
[06b296c] fix struct validation for pfm_event_attr_info_t
[a347a0a] remove duplicate offcore_response_*:l3_miss umask for SNB_EP
[1d57dbe] fix error in pfmlib_is_blacklisted_pmu() with some compilers
[a3012f8] add Intel Haswell-EP alias for offcore_response remote L3 miss
[f7d5075] fix Intel Broadwell-EP OFFCORE_RESPONSE:L3_MISS_REMOTE
[c7e1e2a] update Intel Goldmont support
[b74653d] add support for Intel Goldmont processor
[a31c90e] fix/add offcore_response:l3_miss alias for Intel SNB/IVB/HSW/BDW/SKL
[487937d] IBM Power8 add missing suppored_plm mask initialization

[Andreas Beckmann (anbe)]

On 2016-10-04 08:27, Andreas Beckmann wrote:
>> +# see FEATURE AREAS in dpkg-buildflags(1)
>> +export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-pie
>>
>> This explicitly disables PIE, which is enabled by default on amd64 and
>> s390x in yakkety. Why should we want to disable PIE? There's no
>> explanation in the debian changelog.
>
> PIE wasn't enabled in Debian before that change and IIRC I had some
> build problems on i386 or amd64 or ??? and therefore explicitly didn't
> turn on PIE. I'll have a look at this again within the next weeks.

if I drop ,-pie I get an FTBFS:

[...]
cc -fPIC -DPIC -g -O2 -fdebug-prefix-map=/build/libpfm4-4.7.0+git30-gd422ba2=. -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -Werror -Wextra -Wno-unused-parameter -I. -I/build/libpfm4-4.7.0+git30-gd422ba2/include -DCONFIG_PFMLIB_DEBUG -DCONFIG_PFMLIB_OS_LINUX -g -Wall -Werror -Wextra -Wno-unused-parameter -I. -I/build/libpfm4-4.7.0+git30-gd422ba2/lib/../include -DCONFIG_PFMLIB_DEBUG -DCONFIG_PFMLIB_OS_LINUX -D_REENTRANT -I. -fvisibility=hidden -DCONFIG_PFMLIB_ARCH_X86 -DCONFIG_PFMLIB_ARCH_X86_64 -I. -c pfmlib_amd64_fam15h.c -o pfmlib_amd64_fam15h.lo
cc -g -O2 -fdebug-prefix-map=/build/libpfm4-4.7.0+git30-gd422ba2=. -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -Werror -Wextra -Wno-unused-parameter -I. -I/build/libpfm4-4.7.0+git30-gd422ba2/include -DCONFIG_PFMLIB_DEBUG -DCONFIG_PFMLIB_OS_LINUX -g -Wall -Werror -Wextra -Wno-unused-parameter -I. -I/build/libpfm4-4.7.0+git30-gd422ba2/lib/../include -DCONFIG_PFMLIB_DEBUG -DCONFIG_PFMLIB_OS_LINUX -D_REENTRANT -I. -fvisibility=hidden -DCONFIG_PFMLIB_ARCH_X86 -DCONFIG_PFMLIB_ARCH_X86_64 -I. -fPIE -pie -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname -Wl,libpfm.so.4 -o libpfm.so.4.7.0 pfmlib_common.lo pfmlib_perf_event_pmu.lo pfmlib_perf_event.lo pfmlib_perf_event_raw.lo pfmlib_intel_x86_perf_event.lo pfmlib_amd64_perf_event.lo pfmlib_intel_netburst_perf_event.lo pfmlib_intel_snbep_unc_perf_event.lo pfmlib_amd64.lo pfmlib_intel_core.lo pfmlib_intel_x86.lo pfmlib_intel_x86_arch.lo pfmlib_intel_atom.lo pfmlib_intel_nhm_unc.lo pfmlib_intel_nhm.lo pfmlib_intel_wsm.lo pfmlib_intel_snb.lo pfmlib_intel_snb_unc.lo pfmlib_intel_ivb.lo pfmlib_intel_ivb_unc.lo pfmlib_intel_hsw.lo pfmlib_intel_bdw.lo pfmlib_intel_skl.lo pfmlib_intel_rapl.lo pfmlib_intel_snbep_unc.lo pfmlib_intel_snbep_unc_cbo.lo pfmlib_intel_snbep_unc_ha.lo pfmlib_intel_snbep_unc_imc.lo pfmlib_intel_snbep_unc_pcu.lo pfmlib_intel_snbep_unc_qpi.lo pfmlib_intel_snbep_unc_ubo.lo pfmlib_intel_snbep_unc_r2pcie.lo pfmlib_intel_snbep_unc_r3qpi.lo pfmlib_intel_ivbep_unc_cbo.lo pfmlib_intel_ivbep_unc_ha.lo pfmlib_intel_ivbep_unc_imc.lo pfmlib_intel_ivbep_unc_pcu.lo pfmlib_intel_ivbep_unc_qpi.lo pfmlib_intel_ivbep_unc_ubo.lo pfmlib_intel_ivbep_unc_r2pcie.lo pfmlib_intel_ivbep_unc_r3qpi.lo pfmlib_intel_ivbep_unc_irp.lo pfmlib_intel_hswep_unc_cbo.lo pfmlib_intel_hswep_unc_ha.lo pfmlib_intel_hswep_unc_imc.lo pfmlib_intel_hswep_unc_pcu.lo pfmlib_intel_hswep_unc_qpi.lo pfmlib_intel_hswep_unc_ubo.lo pfmlib_intel_hswep_unc_r2pcie.lo pfmlib_intel_hswep_unc...

[Graham Inggs (ginggs)]

@anbe:
> if I drop ,-pie I get an FTBFS:
I get the same in Ubuntu, building in a PPA.

@vorlon:
Are there any particular commits, listed in comment #4, that you are concerned about?

[Steve Langasek (vorlon)]

On Thu, Oct 06, 2016 at 06:59:10PM -0000, Graham Inggs wrote:
> @anbe:
> > if I drop ,-pie I get an FTBFS:
> I get the same in Ubuntu, building in a PPA.

The package currently in Ubuntu yakkety does not FTBFS in this way with
-fPIE, so this is a regression. And since PIE is a security feature, I'm
not ok with pushing this in the week before release - the regression should
be sorted and the package brought in only once this is resolved.

> /usr/bin/ld: pfmlib_common.lo: relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status

Self-explanatory error message. Where's the compiler commandline for this
pfmlib_common.lo, and why is it missing -fPIE?

[Andreas Beckmann (anbe)]

On 2016-10-06 22:27, Steve Langasek wrote:
>> /usr/bin/ld: pfmlib_common.lo: relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
>> /usr/bin/ld: final link failed: Bad value
>> collect2: error: ld returned 1 exit status
>
> Self-explanatory error message. Where's the compiler commandline for this
> pfmlib_common.lo, and why is it missing -fPIE?

It has both -fPIC and -fPIE (in that order):

cc -fPIC -DPIC -g -O2 -fdebug-prefix-map=/build/libpfm4-4.7.0+git30-gd422ba2=. -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -Werror -Wextra -Wno-unused-parameter -I. -I/build/libpfm4-4.7.0+git30-gd422ba2/include -DCONFIG_PFMLIB_DEBUG -DCONFIG_PFMLIB_OS_LINUX -g -Wall -Werror -Wextra -Wno-unused-parameter -I. -I/build/libpfm4-4.7.0+git30-gd422ba2/lib/../include -DCONFIG_PFMLIB_DEBUG -DCONFIG_PFMLIB_OS_LINUX -D_REENTRANT -I. -fvisibility=hidden -DCONFIG_PFMLIB_ARCH_X86 -DCONFIG_PFMLIB_ARCH_X86_64 -I. -c pfmlib_common.c -o pfmlib_common.lo

Andreas

[Graham Inggs (ginggs)]

I managed to get a bit further with the following patch:

--- a/lib/Makefile
+++ b/lib/Makefile
@@ -33,7 +33,7 @@
 SRCS += pfmlib_perf_event_pmu.c pfmlib_perf_event.c pfmlib_perf_event_raw.c
 endif

-CFLAGS+=-D_REENTRANT -I. -fvisibility=hidden
+CFLAGS+=-D_REENTRANT -I. -fvisibility=hidden -fPIC -DPIC

 #
 # list all library support modules

...but then ended up with a bunch of undefined references building python/src/perfmon_int_wrap.c which seem to be SWIG-related.

[Andreas Beckmann (anbe)]

On 2016-10-07 07:05, Graham Inggs wrote:
> ...but then ended up with a bunch of undefined references building
> python/src/perfmon_int_wrap.c which seem to be SWIG-related.

... because LDFLAGS contains -pie, but we build a .so ...

should be fixed in git

Andreas

[Andreas Beckmann (anbe)]

On 2016-10-07 09:23, Andreas Beckmann wrote:
> should be fixed in git

New version with PIE enabled uploaded to sid, thanks for all your input!

Andreas

[Graham Inggs (ginggs)]

This bug was fixed in the package libpfm4 - 4.7.0+git30-gd422ba2-2
Sponsored for Andreas Beckmann (anbe)

---------------
libpfm4 (4.7.0+git30-gd422ba2-2) unstable; urgency=medium

  * PIE.patch: New, -fPIC needs to override -fPIE.
  * Build with PIE enabled.

 -- Andreas Beckmann <email address hidden> Fri, 07 Oct 2016 15:35:12 +0200

libpfm4 (4.7.0+git30-gd422ba2-1) unstable; urgency=medium

  * New upstream GIT snapshot.
  * Overhaul buildflags usage and enable more hardening.
  * Fix i386 (cross-)build with x86_64 uname.
  * typos.patch: Fix some more typos.

 -- Andreas Beckmann <email address hidden> Mon, 19 Sep 2016 02:21:25 +0200

[Graham Inggs (ginggs)]

This bug was fixed in the package papi - 5.5.0-3
Sponsored for Andreas Beckmann (anbe)

---------------
papi (5.5.0-3) unstable; urgency=medium

  * pie.patch: New, do not build shared libraries with PIE flags.
  * Build with PIE enabled.

 -- Andreas Beckmann <email address hidden> Sat, 08 Oct 2016 13:00:29 +0200

papi (5.5.0-2) unstable; urgency=medium

  * Fix upgrade from jessie with Breaks+Replaces: libpapi5.3.
  * Fix build on hppa, thanks to Helge Deller. (Closes: #838321)

 -- Andreas Beckmann <email address hidden> Fri, 23 Sep 2016 12:14:28 +0200

papi (5.5.0-1) unstable; urgency=medium

  * New upstream release.
  * Refresh patches.
  * fix-typos.patch: Fix some more typos.
  * Bump libpfm4-dev B-D to >= 4.7.0+git30.
  * Restore reproducibility.
  * Upload to unstable.

 -- Andreas Beckmann <email address hidden> Mon, 19 Sep 2016 09:40:23 +0200

papi (5.4.3-4) experimental; urgency=medium

  * Upstream has clarified what composes the library API.
  * Soversion change: 5.4.3 -> 5. Rename package libpapi5.4.3 to libpapi5.
  * Drop support for changing the Soversion frequently.
  * Overhaul buildflags usage and enable more hardening.
  * Upload to experimental.

 -- Andreas Beckmann <email address hidden> Sun, 29 May 2016 10:39:59 +0200