xen ftbfs in disco

[Matthias Klose]

[Impact]
Warning message causes FTBFS when configured to treat warnings as errors (edit: there are actually multiple failures due to new gcc-8 and gcc-9 warnings which are treated as errors). This bug report is used to track all the needed changes to allow compiling again in Disco and later.

IMPORTANT NOTE(!): Albeit compiling, there is still an unresolved problem when compiling with gcc-9 (and possibly newer) which produces a Xen hypervisor which is unbootable (the host immediately reboots after start (no message) as if hitting a triple fault.
So as a quick solution to get a working Xen hypervisor which works around bug #1851091, the only option is to do the upload to Disco and copy forward with binaries.

[Test Case]
$ apt-get source xen
(...)
$ cd xen*
$ sudo apt-get build-dep xen
(...)
$ ./configure
(...)
$ make
(...)
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 36 and 547 bytes into a destination of size 512
     return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        __bos (__s), __fmt, __va_arg_pack ());
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[4]: *** [/home/ubuntu/xen-4.9.2/tools/misc/../../tools/Rules.mk:220: xen-lowmemd.o] Error 1
make[4]: Leaving directory '/home/ubuntu/xen-4.9.2/tools/misc'
$ echo $?
2

[Regression Potential]
Changes string formatting of an error message, which should be
transparent to users, but things to watch for would relate to xen's
error message logs.

[Fix]
A simple tweak to snprintf() formatting from upstream would prevent
the theoretical buffer overflow being warned about. This needs applied
to disco and eoan; bionic and earlier are not affected.

[Discussion]

[Original Report]
https://launchpadlibrarian.net/417924491/buildlog_ubuntu-disco-amd64.xen_4.9.2-0ubuntu2_BUILDING.txt.gz

gcc -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable -Wno-unused-local-typedefs -O2 -fomit-frame-pointer -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF .xen-lowmemd.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -Werror -include /<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/config.h -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/libs/evtchn/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/libxc/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/libs/toollog/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/libs/foreignmemory/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/libs/devicemodel/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -D__XEN_TOOLS__ -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/xenstore/include -I/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/include -c -o xen-lowmemd.o xen-lowmemd.c
xen-lowmemd.c: In function ‘handle_low_mem’:
xen-lowmemd.c:80:55: error: ‘%s’ directive output may be truncated writing up to 511 bytes into a region of size 489 [-Werror=format-truncation=]
         snprintf(error, BUFSZ,"Failed to write target %s to xenstore", data);
                                                       ^~ ~~~~
In file included from /usr/include/stdio.h:867,
                 from xen-lowmemd.c:6:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note: ‘__builtin___snprintf_chk’ output between 36 and 547 bytes into a destination of size 512
   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        __bos (__s), __fmt, __va_arg_pack ());
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[5]: *** [/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc/../../tools/Rules.mk:220: xen-lowmemd.o] Error 1
make[5]: Leaving directory '/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/misc'
make[4]: *** [/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/../tools/Rules.mk:244: subdir-all-misc] Error 2
make[4]: Leaving directory '/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools'
make[3]: *** [/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools/../tools/Rules.mk:239: subdirs-all] Error 2
make[3]: Leaving directory '/<<PKGBUILDDIR>>/debian/build/build-utils_amd64/tools'
make[2]: *** [debian/rules.real:111: debian/stamps/build-utils_amd64] Error 2
make[2]: Leaving directory '/<<PKGBUILDDIR>>'
make[1]: *** [debian/rules.gen:66: build-arch_amd64_real] Error 2
make[1]: Leaving directory '/<<PKGBUILDDIR>>'

[Matthias Klose (doko)]

also ftbfs in cosmic:
https://launchpadlibrarian.net/418304930/buildlog_ubuntu-cosmic-amd64.xen_4.9.2-0ubuntu2_BUILDING.txt.gz

[Matthias Klose (doko)]

also ftbfs in eoan

[Bryce Harrington (bryce)]

Looks like this is fixed upstream with commit 27751d89248c8c5eef6d8b56eb8f7d2084145080.

Upstream patch is attached; this would need applied to cosmic, disco, and eoan.

[Bryce Harrington (bryce)]

Dropping cosmic task since cosmic reached end of support this month.

[Bryce Harrington (bryce)]

Branches with the packaging changes:
    ~bryce/ubuntu/+source/xen:xen-sru-1823441-disco
    ~bryce/ubuntu/+source/xen:xen-sru-1823441-eoan

PPA with proposed fix for Disco and Eoan:

     https://launchpad.net/~bryce/+archive/ubuntu/xen-sru-1823441
     ppa:bryce/xen-sru-1823441

[Bryce Harrington (bryce)]

Looks like it built locally for me with this patch when I had gcc8 installed, but in the ppa it's defaulting to gcc9 for some reason, and fails to build.

[Rafael David Tinoco (rafaeldtinoco)]

I'm also getting other building errors such as:

dsdt_anycpu.c:7495:28: error: ‘dsdt_anycpu’ undeclared here (not in a function); did you mean ‘dsdt_anycpu_len’?
 int dsdt_anycpu_len=sizeof(dsdt_anycpu);
                            ^~~~~~~~~~~
                            dsdt_anycpu_len

 int dsdt_15cpu_len=sizeof(dsdt_15cpu);
                           ^~~~~~~~~~
                           dsdt_15cpu_len
make[8]: *** [/home/inaddy/work/sources/oldubuntu/disco/xen-4.9.2/tools/firmware/hvmloader/../../../tools/Rules.mk:220: dsdt_15cpu.o] Error 1

../../libacpi/build.c:392:49: error: ‘ssdt_s3’ undeclared (first use in this function); did you mean ‘ssdt’?
         ssdt = ctxt->mem_ops.alloc(ctxt, sizeof(ssdt_s3), 16);
                                                 ^~~~~~~
                                                 ssdt

Maybe we should fix them all using this LP bug ?

o/

[Timo Aaltonen (tjaalton)]

the SRU upload was targeted for disco, while the changelog says it's about eoan.. and the version is bumped to -0ubuntu4 skipping 3 -> rejected

both disco and eoan share the same xen version right now, so care should be used with the sru version

[Timo Aaltonen (tjaalton)]

An upload of xen to disco-proposed has been rejected from the upload queue for the following reason: "cruft in the package (d/changelog.orig), and could use -0ubuntu3 as the version".

[Timo Aaltonen (tjaalton)]

Hello Matthias, or anyone else affected,

Accepted xen into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/xen/4.9.2-0ubuntu5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Launchpad Janitor (janitor)]

This bug was fixed in the package xen - 4.9.2-0ubuntu6

---------------
xen (4.9.2-0ubuntu6) focal; urgency=medium

  * Build-depend on python2-dev.
  * Depend on python2.
  * Build using python2.
  * Build-depend on lmodern.

 -- Matthias Klose <email address hidden> Mon, 13 Jan 2020 14:51:35 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package xen - 4.9.2-0ubuntu5

---------------
xen (4.9.2-0ubuntu5) disco; urgency=medium

  * Fix FTBS in Eoan (LP: #1823441). Except the last two changes, these are
    all cherry picks from Xen upstream to handle gcc8 and gcc9 changes.
    - d/p/x86-e820-fix-build-with-gcc9.patch
    - d/p/x86-IO-APIC-fix-build-with-gcc9.patch
    - d/p/trace-fix-build-with-gcc9.patch
    - d/p/tools-libxc-fix-strncpy-size.patch
    - d/p/tools-misc-fix-hypothetical-buffer-overflow-in-xen-l.patch
    - d/p/tools-xentop-replace-use-of-deprecated-vwprintw.patch
    - d/p/tools-xenpmd-fix-possible-0-truncation.patch
    - d/p/xenpmd-make-32-bit-gcc-8.1-non-debug-build-work.patch
    - d/p/libacpi-fixes-for-iasl-20180427.patch
    - d/p/tools-blktap2-fix-possible-0-truncation.patch
    - d/p/tools-blktap2-fix-hypothetical-buffer-overflow.patch
    - d/p/libxl-arm-Fix-build-on-arm64-acpi-w-gcc-8.2.patch
    - d/p/ubuntu/flags-fcs-protect-none.patch
    - d/p/ubuntu/strip-note-gnu-property.patch
  * Fix decode failed panics with v5.2+ kernels (LP: #1851091)
    - d/p/0001-lz4-refine-commit-9143a6c55ef7-for-the-64-bit-case.patch
    - d/p/0002-lz4-pull-out-constant-tables.patch
    - d/p/0003-lz4-fix-system-halt-at-boot-kernel-on-x86_64.patch

 -- Stefan Bader <email address hidden> Wed, 11 Dec 2019 17:23:34 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for xen has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Andreas Hasenack (ahasenack)]

The disco sru was released before eoan's?

[Launchpad Janitor (janitor)]

This bug was fixed in the package xen - 4.9.2-0ubuntu5

---------------
xen (4.9.2-0ubuntu5) disco; urgency=medium

  * Fix FTBS in Eoan (LP: #1823441). Except the last two changes, these are
    all cherry picks from Xen upstream to handle gcc8 and gcc9 changes.
    - d/p/x86-e820-fix-build-with-gcc9.patch
    - d/p/x86-IO-APIC-fix-build-with-gcc9.patch
    - d/p/trace-fix-build-with-gcc9.patch
    - d/p/tools-libxc-fix-strncpy-size.patch
    - d/p/tools-misc-fix-hypothetical-buffer-overflow-in-xen-l.patch
    - d/p/tools-xentop-replace-use-of-deprecated-vwprintw.patch
    - d/p/tools-xenpmd-fix-possible-0-truncation.patch
    - d/p/xenpmd-make-32-bit-gcc-8.1-non-debug-build-work.patch
    - d/p/libacpi-fixes-for-iasl-20180427.patch
    - d/p/tools-blktap2-fix-possible-0-truncation.patch
    - d/p/tools-blktap2-fix-hypothetical-buffer-overflow.patch
    - d/p/libxl-arm-Fix-build-on-arm64-acpi-w-gcc-8.2.patch
    - d/p/ubuntu/flags-fcs-protect-none.patch
    - d/p/ubuntu/strip-note-gnu-property.patch
  * Fix decode failed panics with v5.2+ kernels (LP: #1851091)
    - d/p/0001-lz4-refine-commit-9143a6c55ef7-for-the-64-bit-case.patch
    - d/p/0002-lz4-pull-out-constant-tables.patch
    - d/p/0003-lz4-fix-system-halt-at-boot-kernel-on-x86_64.patch

 -- Stefan Bader <email address hidden> Wed, 11 Dec 2019 17:23:34 +0100

[Andy Whitcroft (apw)]

We don't specifically prevent such mal-ordered releases, we prefer to not do them, but as long as someone has taken responsibility for ensuring they are released we allow such things. I have now released it in eoan as well.